広告掲載
MCP.so
ログイン

Crawl4AI MCP Server

@seungwonme

リポジトリ

基本情報

カテゴリ

その他

トランスポート

stdio

作者

seungwonme

Crawl4AI MCP Server について

概要はまだありません

#crawl4ai#mcp-server#web-scraping

Crawl4AI MCP Server の設定

以下の設定を使って、このサーバーを MCP 対応クライアントに追加してください。

{
  "mcpServers": {
    "crawl4ai-mcp-server-seungwonme": {
      "command": "uv",
      "args": [
        "sync"
      ]
    }
  }
}

Crawl4AI MCP Server のツール

ツール一覧はまだありません

この掲載には表示できるツール一覧がまだありません。サーバーにツールがないという意味ではありません。対応機能はプロジェクトのドキュメントをご確認ください。

ツールは README から自動的に抽出されます。メンテナーは ## Tools という見出しの下に記載することで、このタブに反映できます。

概要

What is Crawl4AI MCP Server?

Crawl4AI Documentation Crawler is a CLI tool that crawls documentation websites using the Crawl4AI library and converts pages to Markdown files. It supports single‑page and recursive deep crawling with URL prefix filtering, automatic navigation/footer removal, and domain‑based output directory generation.

How to use Crawl4AI MCP Server?

Install dependencies with uv sync. Use the CLI command uv run cli.py crawl <url> [options] to crawl a page or entire site. Specify --recursive for deep crawling, -o to save to a directory, and --prefix to filter URLs. Run uv run cli.py config-list to see available preset configurations.

Key features of Crawl4AI MCP Server

  • Converts web pages to clean Markdown files
  • Recursive deep crawl with configurable depth and page limits
  • URL prefix filter for selective crawling of specific paths
  • Automatic removal of navigation and footer content
  • Domain‑based directory organization for output files
  • Asynchronous crawling using AsyncIO for performance

Use cases of Crawl4AI MCP Server

  • Crawl and archive a full product documentation site as Markdown
  • Extract specific sections of a large documentation site using prefix filtering
  • Create a local offline backup of API references or developer guides
  • Convert multi‑page documentation into structured file trees for further processing

よくある質問

1

What is the difference between single‑page and deep crawl mode?

Single‑page mode crawls only the given URL and outputs Markdown to the console or a single file. Deep crawl mode recursively follows links on the same domain to spider the entire site, saving pages in a directory tree.

2

What are the runtime and dependency requirements?

The tool requires Python 3.13+ and uses `uv` for dependency management. It depends on Crawl4AI, Typer, and AsyncIO.

3

Where does the crawled data get saved?

If no output directory is specified, Markdown appears on the console. With `-o`, data is saved to a directory named after the domain (e.g., `docs_crawl4ai_com/`). Deep crawl mode always saves to a directory.

4

Are there any limits on crawling?

Yes: for deep crawl, you can set `--max-pages` (default 100) and `--max-depth` (default 2). Only same‑domain pages are crawled automatically.

5

Does the tool support authentication or private sites?

The README does not mention any authentication support or transport protocols. It is designed for public documentation sites only.

コメント

「その他」の他のコンテンツ

Handl

Handl speaks the Model Context Protocol. Your whole billing operation — projects, invoices, clients, reports, approvals — as tools any AI assistant can call, inside your autonomy rules.

EverArt

modelcontextprotocol

Model Context Protocol Servers

88.1K

Awesome Mlops

visenger

A curated list of references for MLOps

14.0K

Yuppsy

Yuppsy MCP: build and publish real estate websites from your AI assistant. Yuppsy lets your AI assistant create live websites for real estate agents and individual property listings. You don't need a designer or any code

Figs, Relationship Expert

Figs O'Sullivan, LMFT

Couples therapist Figs O'Sullivan (LMFT, 17 years, endorsed by Sue Johnson, creator of EFT) inside your AI. Ask relationship questions answered from his clinical writing, name the pattern in a fight, get a three-step rep

広告掲載