広告掲載
MCP.so
ログイン
C

Crawl4ai Rag MCP Server

@coleam00

リポジトリ

基本情報

カテゴリ

メモリとナレッジ

トランスポート

stdio

作者

coleam00

投稿者

Tariq Forde

Crawl4ai Rag MCP Server について

概要はまだありません

#crawl4ai#web-crawling#ai-coding-assistant

Crawl4ai Rag MCP Server の設定

以下の設定を使って、このサーバーを MCP 対応クライアントに追加してください。

{
  "mcpServers": {
    "crawl4ai-rag": {
      "command": "python",
      "args": [
        "path/to/crawl4ai-mcp/src/crawl4ai_mcp.py"
      ],
      "env": {
        "TRANSPORT": "stdio",
        "OPENAI_API_KEY": "your_openai_api_key",
        "SUPABASE_URL": "your_supabase_url",
        "SUPABASE_SERVICE_KEY": "your_supabase_service_key",
        "USE_KNOWLEDGE_GRAPH": "false",
        "NEO4J_URI": "bolt://localhost:7687",
        "NEO4J_USER": "neo4j",
        "NEO4J_PASSWORD": "your_neo4j_password"
      }
    }
  }
}

Crawl4ai Rag MCP Server のツール

ツール一覧はまだありません

この掲載には表示できるツール一覧がまだありません。サーバーにツールがないという意味ではありません。対応機能はプロジェクトのドキュメントをご確認ください。

ツールは README から自動的に抽出されます。メンテナーは ## Tools という見出しの下に記載することで、このタブに反映できます。

概要

What is Crawl4ai Rag?

A Model Context Protocol (MCP) server that integrates Crawl4AI web crawling with Supabase vector storage to provide AI agents and coding assistants with web crawling and RAG (Retrieval-Augmented Generation) capabilities.

How to use Crawl4ai Rag?

Install via Docker or uv (Python 3.12+), configure a .env file with Supabase and OpenAI API key, set up a Supabase database with pgvector, and optionally set up Neo4j for knowledge graph features. The server exposes MCP tools for crawling, searching, and querying crawled content.

Key features of Crawl4ai Rag

  • Smart URL detection (regular pages, sitemaps, text files)
  • Recursive crawling following internal links
  • Parallel processing for efficient multi-page crawling
  • Content chunking by headers and size
  • Vector search with optional source filtering
  • Advanced RAG strategies (contextual embeddings, hybrid search, agentic RAG, reranking, knowledge graph)

Use cases of Crawl4ai Rag

  • AI coding assistants scraping and indexing documentation for code-aware RAG
  • Building a knowledge engine for AI agents to crawl and query websites
  • Performing semantic search over previously crawled content with source filtering
  • Detecting AI hallucinations by validating code against a repository knowledge graph

よくある質問

1

What dependencies does Crawl4ai Rag require?

Requires Docker or Python 3.12+, a Supabase project (for RAG), and an OpenAI API key. Neo4j is optional for knowledge graph functionality.

2

How are RAG strategies configured?

Each strategy (contextual embeddings, hybrid search, agentic RAG, reranking, knowledge graph) is enabled via a `USE_*` environment variable set to `"true"` in the `.env` file. All default to `"false"`.

3

Where is crawled content stored?

Content is stored in a Supabase database with the pgvector extension. A separate table is created for agentic RAG code examples if enabled. Knowledge graph data is stored in a Neo4j graph database.

4

What transport does the server use?

The server uses SSE (Server-Sent Events) transport by default, configurable via the `TRANSPORT` environment variable.

5

Are there known limitations or planned improvements?

The repository is a testbed for integration into Archon; known areas for improvement include supporting multiple embedding models, local Ollama execution, faster crawling/indexing, and enhanced chunking strategies.

コメント

「メモリとナレッジ」の他のコンテンツ

Context7 MCP - Up-to-date Docs For Any Cursor Prompt

upstash

Context7 Platform -- Up-to-date code documentation for LLMs and AI code editors

58.7K

Sensefold

SenseFold Labs

Personal context for every AI: search, read, and write back to your private Markdown library of articles, threads, PDFs, notes, and captured ChatGPT/Claude/Gemini/Grok conversations. Remote server, OAuth 2.1 paste-and-au

Leviathan Systems Public Content MCP

Leviathan Systems

Search published GPU deployment, rack integration and structured cabling resources; read canonical documents and Leviathan Systems company service scope. Anonymous, read-only Streamable HTTP MCP. No credentials required.

Groundwork

Rarefied-Earth

Groundwork: company memory for your AI tools. Docs public. Source closed. Public proof needs no account.

1

priostack

ideaswave

Open-source Python SDK & MCP interface for Priostack Agent Context Network (ACN). Zero-setup, autonomous memory orchestration for AI agents.

広告掲載