crawl4ai-mcp

Name: crawl4ai-mcp
Author: wyattowalsh

@wyattowalsh

代码仓库主页

关于 crawl4ai-mcp

MCP (Model Context Protocol) server wrapping Crawl4AI library as functions using Python

#python #agent #crawler #scraper #mcp #crawling #python3 #scrape #uv #uvx #agentic-workflow #agentic-ai #crawl4ai #model-context-protocol #mcp-servers #mcp-server #model-context-protocol-servers

基本信息

分类

其他

许可证

MIT

运行时

python

传输方式

stdio

发布者

wyattowalsh

配置

使用下面的配置,将此服务器添加到你的 MCP 客户端。

{
  "mcpServers": {
    "crawl4ai-mcp": {
      "command": "docker",
      "args": [
        "build",
        "-t",
        "mcp-crawl4ai",
        "."
      ]
    }
  }
}

工具

未检测到工具

工具是从 README 中自动提取的。维护者可以在 ## Tools 标题下列出工具,即可填充这部分内容。

概览

What is crawl4ai-mcp?

crawl4ai-mcp is a Model Context Protocol (MCP) server that provides AI systems with access to the live web. Built on FastMCP v3 and Crawl4AI, it exposes 4 tools, 2 resources, and 3 prompts through a standardized MCP interface, backed by a lifespan-managed headless Chromium browser.

How to use crawl4ai-mcp?

Install via pip install mcp-crawl4ai or uv add mcp-crawl4ai, then run mcp-crawl4ai for stdio transport (default) or mcp-crawl4ai --transport http --port 8000 for HTTP. For Claude Desktop, add to MCP settings with the command mcp-crawl4ai and args ["--transport", "stdio"].

Key features of crawl4ai-mcp

Full MCP compliance via FastMCP v3 with tool annotations
4 focused tools for scrape, crawl, session lifecycle, and artifacts
3 prompts for common LLM workflows (summarize, extract schema, compare)
2 resources exposing server configuration and version info
Headless Chromium managed as a lifespan singleton
Multiple transports: stdio (default) and Streamable HTTP
LLM-optimized output in markdown, cleaned HTML, raw HTML, or plain text
Auto browser setup and detection of missing Playwright browsers

Use cases of crawl4ai-mcp

AI agents that need to scrape or crawl web pages for real-time information
Summarizing a page with a specified focus using the summarize_page prompt
Building a CSS extraction schema for structured data from a URL
Comparing two web pages and producing a structured comparison
Deep traversal of a website using recursive BFS/DFS from a single seed

FAQ from crawl4ai-mcp

What is the difference between crawl4ai-mcp and using Crawl4AI directly?

crawl4ai-mcp wraps Crawl4AI in an MCP server with only 2 runtime dependencies (fastmcp and crawl4ai), providing a standardized interface for AI tools and LLM clients. It adds session management, resource endpoints, and prompt templates.

What are the runtime dependencies of crawl4ai-mcp?

The server requires Python 3.13+ and has exactly two runtime dependencies: fastmcp and crawl4ai. Playwright browsers are needed for headless Chromium but are installed automatically or via mcp-crawl4ai --setup.

Does crawl4ai-mcp require a browser to be installed?

Yes, it uses headless Chromium. The server auto-detects missing Playwright browsers and attempts to install them on first startup, or you can run mcp-crawl4ai --setup manually. The Docker image includes browsers pre-installed.

What transports does crawl4ai-mcp support?

Stdio (default) for desktop clients like Claude Desktop and Cursor, and HTTP (Streamable) for remote access. HTTP binds to 127.0.0.1:8000 by default; use --host and a reverse proxy for external exposure.

Are there any known limits in crawl4ai-mcp?

The scrape tool accepts up to 20 URLs in a bounded list. The crawl tool supports both list traversal (bounded) and deep traversal (recursive BFS/DFS) from a single seed. There are configurable limits exposed via the config://server resource.

其他分类下的更多 MCP 服务器

XcodeBuildMCP

cameroncooke

A Model Context Protocol (MCP) server and CLI that provides tools for agent use when working on iOS and macOS projects.

6.0K

Peekaboo MCP – lightning-fast macOS screenshots for AI agents

steipete

Peekaboo is a macOS CLI & optional MCP server that enables AI agents to capture screenshots of applications, or the entire system, with optional visual question answering through local or remote AI models.

4.8K

IDA Pro MCP

mrexodia

AI-powered reverse engineering assistant that bridges IDA Pro with language models through MCP.

9.9K

ACI: Open-Source Infra to Power Unified MCP Servers

aipotheosis-labs

ACI.dev is the open source tool-calling platform that hooks up 600+ tools into any agentic IDE or custom AI agent through direct function calling or a unified MCP server. The birthplace of VibeOps.