Crawl4 MCP Server

@ShiDuLin

代码仓库

基本信息

分类

其他

传输方式

stdio

作者

ShiDuLin

关于 Crawl4 MCP Server

本项目是一个crawl4ai 的爬虫MCP SERVER，提供高级网络爬虫。使用此 MCP SERVER，您可以抓取任何内容，将内容保存为本地markdown文件，然后在任何地方将该知识用于 RAG。

#crawl4ai #web-scraper #data-collection

Crawl4 MCP Server 配置

使用下面的配置,将此服务器添加到你的 MCP 客户端。

{
  "mcpServers": {
    "crawl4-mcp": {
      "command": "uv",
      "args": [
        "venv",
        "-p",
        "3.12"
      ]
    }
  }
}

Crawl4 MCP Server 工具

工具列表暂未提供

当前收录暂时没有可展示的工具列表，不代表服务没有工具。请查阅项目文档了解支持的能力。

工具是从 README 中自动提取的。维护者可以在 ## Tools 标题下列出工具,即可填充这部分内容。

概览

What is crawl4-mcp?

crawl4-mcp is an MCP server built on crawl4ai that provides advanced web crawling capabilities. It allows users to crawl any web content, save it as local markdown files, and use that knowledge anywhere for RAG (Retrieval-Augmented Generation).

How to use crawl4-mcp?

Install by cloning the repository, creating a Python 3.12 virtual environment with uv, running uv sync and crawl4ai-setup, then creating a .env file. Start the server with uv run src/crawl4ai_mcp.py. The server listens on the configured host and port via SSE transport. Integrate with any MCP client using transport: sse and url: http://localhost:8051/sse.

Key features of crawl4-mcp

Crawls any web content with advanced crawl4ai engine
Saves content as local markdown files
Output ready for RAG (Retrieval-Augmented Generation)
SSE-based MCP server integration
Simple Python 3.12+ setup with uv

Use cases of crawl4-mcp

Building a local knowledge base from web pages for RAG pipelines
Archiving website content as markdown for offline reference
Feeding crawled data into AI assistants or document retrieval systems

常见问题

What are the environment requirements?

Python 3.12 or higher and the uv package manager are required.

How do I install crawl4-mcp?

Clone the repository, create a Python 3.12 virtual environment with uv, run `uv sync` and `crawl4ai-setup`, then create a `.env` file.

How do I run the server?

Execute `uv run src/crawl4ai_mcp.py` in the activated virtual environment.

How do I integrate crawl4-mcp with an MCP client?

Configure the client with transport type `sse` and the URL `http://localhost:8051/sse` (adjust host/port as needed).

What happens to the crawled data?

The content is saved as local markdown files, ready to be used for RAG or other applications.

其他分类下的更多 MCP 服务器

Yuppsy

Yuppsy MCP: build and publish real estate websites from your AI assistant. Yuppsy lets your AI assistant create live websites for real estate agents and individual property listings. You don't need a designer or any code

Senso MCP for Joomla

sensomedia

Senso MCP turns a Joomla 5 or 6 site into a remote MCP server. It runs inside Joomla itself (no extra server, no Node.js bridge) and is secured with OAuth 2.1 and Joomla's own ACL. The free pack lets an AI assistant crea

Volley.sh

Volley

A good sequence diagram can solve almost any communication problem in engineering. Volley lets your agent write them as plain text, with the real request and response payloads on every arrow. It checks its own syntax wi