Crawl4ai MCP Server

Name: Crawl4ai MCP Server
Author: Kirill812

@Kirill812

リポジトリ

Crawl4ai MCP Server について

A Model Context Protocol server that provides web crawling capabilities using Crawl4ai

基本情報

カテゴリ

その他

ライセンス

MIT license

ランタイム

node

トランスポート

stdio

公開者

Kirill812

設定

以下の設定を使って、このサーバーを MCP 対応クライアントに追加してください。

{
  "mcpServers": {
    "crawl4ai": {
      "command": "node",
      "args": [
        "/path/to/crawl4ai-mcp-server/build/index.js"
      ],
      "env": {
        "CRAWL4AI_API_URL": "",
        "CRAWL4AI_AUTH_TOKEN": ""
      }
    }
  }
}

ツール

Crawl web pages and get markdown content with citations

概要

What is Crawl4ai MCP Server?

A MCP server that provides web crawling capabilities using crawl4ai, returning markdown content with citations for use by LLMs. It is for developers building AI applications that need to fetch and structure web page content via the Model Context Protocol.

How to use Crawl4ai MCP Server?

Install dependencies with npm install, build with npm run build, then add the server configuration to your MCP client environment, setting CRAWL4AI_API_URL (optional) and CRAWL4AI_AUTH_TOKEN (optional). Invoke the crawl_urls tool with a list of URLs to crawl.

Key features of Crawl4ai MCP Server

Single tool crawl_urls accepts a list of URLs
Returns markdown content with numbered citations per URL
Multiple URLs are separated by horizontal rules
Automatically handles retries with different user agents
Requires a running crawl4ai API instance (Docker deployment)

Use cases of Crawl4ai MCP Server

Providing up-to-date web content as context for LLM prompts
Scraping documentation or articles for summarization
Aggregating content from multiple URLs into a single markdown response

FAQ from Crawl4ai MCP Server

What is crawl4ai and do I need it?

Crawl4ai is an external web crawling service. You must have a running crawl4ai instance, typically deployed via Docker, for this server to work.

What are the runtime requirements?

Node.js and a running crawl4ai API instance. No other dependencies are needed beyond the npm packages.

Where does the web page data live?

The server does not store data; it fetches content from each URL via the crawl4ai API and returns it directly to the MCP client.

Are there any known limits or errors?

Timeouts may occur with too many URLs per request. The service handles retries automatically, but you can reduce the number of URLs if problems persist.