広告掲載
MCP.so
ログイン

Crawl4 MCP Server

@ShiDuLin

リポジトリ

基本情報

カテゴリ

その他

トランスポート

stdio

作者

ShiDuLin

Crawl4 MCP Server について

本项目是一个crawl4ai 的爬虫MCP SERVER，提供高级网络爬虫。使用此 MCP SERVER，您可以抓取任何内容，将内容保存为本地markdown文件，然后在任何地方将该知识用于 RAG。

#crawl4ai#web-scraper#data-collection

Crawl4 MCP Server の設定

以下の設定を使って、このサーバーを MCP 対応クライアントに追加してください。

{
  "mcpServers": {
    "crawl4-mcp": {
      "command": "uv",
      "args": [
        "venv",
        "-p",
        "3.12"
      ]
    }
  }
}

Crawl4 MCP Server のツール

ツール一覧はまだありません

この掲載には表示できるツール一覧がまだありません。サーバーにツールがないという意味ではありません。対応機能はプロジェクトのドキュメントをご確認ください。

ツールは README から自動的に抽出されます。メンテナーは ## Tools という見出しの下に記載することで、このタブに反映できます。

概要

What is crawl4-mcp?

crawl4-mcp is an MCP server built on crawl4ai that provides advanced web crawling capabilities. It allows users to crawl any web content, save it as local markdown files, and use that knowledge anywhere for RAG (Retrieval-Augmented Generation).

How to use crawl4-mcp?

Install by cloning the repository, creating a Python 3.12 virtual environment with uv, running uv sync and crawl4ai-setup, then creating a .env file. Start the server with uv run src/crawl4ai_mcp.py. The server listens on the configured host and port via SSE transport. Integrate with any MCP client using transport: sse and url: http://localhost:8051/sse.

Key features of crawl4-mcp

  • Crawls any web content with advanced crawl4ai engine
  • Saves content as local markdown files
  • Output ready for RAG (Retrieval-Augmented Generation)
  • SSE-based MCP server integration
  • Simple Python 3.12+ setup with uv

Use cases of crawl4-mcp

  • Building a local knowledge base from web pages for RAG pipelines
  • Archiving website content as markdown for offline reference
  • Feeding crawled data into AI assistants or document retrieval systems

よくある質問

1

What are the environment requirements?

Python 3.12 or higher and the uv package manager are required.

2

How do I install crawl4-mcp?

Clone the repository, create a Python 3.12 virtual environment with uv, run `uv sync` and `crawl4ai-setup`, then create a `.env` file.

3

How do I run the server?

Execute `uv run src/crawl4ai_mcp.py` in the activated virtual environment.

4

How do I integrate crawl4-mcp with an MCP client?

Configure the client with transport type `sse` and the URL `http://localhost:8051/sse` (adjust host/port as needed).

5

What happens to the crawled data?

The content is saved as local markdown files, ready to be used for RAG or other applications.

コメント

「その他」の他のコンテンツ

Volley.sh

Volley

A good sequence diagram can solve almost any communication problem in engineering. Volley lets your agent write them as plain text, with the real request and response payloads on every arrow. It checks its own syntax wi

Reactive Resume

amruthpillai

A one-of-a-kind resume builder that keeps your privacy in mind. Completely secure, customizable, portable, open-source and free forever. Try it out today!

39.5K

Codelf

unbug

A search tool helps dev to solve the naming things problem.

14.2K

Klarix Intelligence Engine

Klarix

B2B competitive intelligence, live sales battlecards, 5-axis ICP prospect fit scoring, and tech stack teardowns over MCP and REST with cited public-web evidence.

Awesome Mlops

visenger

A curated list of references for MLOps

14.0K
広告掲載