Crawl4ai Middleware

Name: Crawl4ai Middleware
Author: Frankieli123

@Frankieli123

リポジトリ

Crawl4ai Middleware について

概要はまだありません

#crawl4ai #web-scraping #middleware

基本情報

カテゴリ

その他

トランスポート

stdio

公開者

Frankieli123

投稿者

設定

以下の設定を使って、このサーバーを MCP 対応クライアントに追加してください。

{
  "mcpServers": {
    "crawl4ai": {
      "command": "python",
      "args": [
        "E:/APP/craw14ai-middleware/crawl4ai_server.py"
      ]
    }
  }
}

ツール

ツールは検出されませんでした

ツールは README から自動的に抽出されます。メンテナーは ## Tools という見出しの下に記載することで、このタブに反映できます。

概要

What is Crawl4ai Middleware?

Crawl4ai Middleware is an MCP (Model Context Protocol) server that integrates crawl4ai's web scraping capabilities into MCP-compatible applications like Cursor IDE. It provides tools to create crawl tasks, retrieve results by task ID, list saved files, and read their contents.

How to use Crawl4ai Middleware?

Install dependencies with pip install mcp[cli] httpx python-dotenv, configure environment variables (API base URL and key), then add the server configuration to your MCP client (e.g., Cursor IDE's settings.json) pointing to the crawl4ai_server.py script.

Key features of Crawl4ai Middleware

Provides four MCP tools for web scraping workflows
Saves scraped content locally to avoid display size limits
Supports environment variable configuration for API and output directory
Allows retrieval of crawl results by task ID
Lists and reads saved result files
Works with Cursor IDE and other MCP-enabled applications

Use cases of Crawl4ai Middleware

Scrape a web page and retrieve its content directly in Cursor IDE
Manage multiple crawl tasks with persistent file storage
Review previously crawled results offline at any time
Integrate web scraping into AI agent workflows via MCP

FAQ from Crawl4ai Middleware

What tools does Crawl4ai Middleware provide?

It provides create_crawl_task, get_crawl_result, list_saved_results, and read_saved_result.

How are crawled files stored?

Files are saved in a url folder next to the server script, named extracted_domain_taskID_timestamp.txt.

What dependencies are required?

Install mcp[cli], httpx, and python-dotenv via pip.

How do I configure the server?

Set environment variables CRAWL4AI_API_BASE, CRAWL4AI_API_KEY, optionally OUTPUT_DIR and LOG_LEVEL. Default API base is http://192.168.31.12:11235 and key sk-3180623.