广告合作
MCP.so
登录
C

Crawl4ai Rag MCP Server

@coleam00

代码仓库

基本信息

分类

记忆与知识

传输方式

stdio

作者

coleam00

提交者

Tariq Forde

关于 Crawl4ai Rag MCP Server

暂无概览

#crawl4ai#web-crawling#ai-coding-assistant

Crawl4ai Rag MCP Server 配置

使用下面的配置,将此服务器添加到你的 MCP 客户端。

{
  "mcpServers": {
    "crawl4ai-rag": {
      "command": "python",
      "args": [
        "path/to/crawl4ai-mcp/src/crawl4ai_mcp.py"
      ],
      "env": {
        "TRANSPORT": "stdio",
        "OPENAI_API_KEY": "your_openai_api_key",
        "SUPABASE_URL": "your_supabase_url",
        "SUPABASE_SERVICE_KEY": "your_supabase_service_key",
        "USE_KNOWLEDGE_GRAPH": "false",
        "NEO4J_URI": "bolt://localhost:7687",
        "NEO4J_USER": "neo4j",
        "NEO4J_PASSWORD": "your_neo4j_password"
      }
    }
  }
}

Crawl4ai Rag MCP Server 工具

工具列表暂未提供

当前收录暂时没有可展示的工具列表，不代表服务没有工具。请查阅项目文档了解支持的能力。

工具是从 README 中自动提取的。维护者可以在 ## Tools 标题下列出工具,即可填充这部分内容。

概览

What is Crawl4ai Rag?

A Model Context Protocol (MCP) server that integrates Crawl4AI web crawling with Supabase vector storage to provide AI agents and coding assistants with web crawling and RAG (Retrieval-Augmented Generation) capabilities.

How to use Crawl4ai Rag?

Install via Docker or uv (Python 3.12+), configure a .env file with Supabase and OpenAI API key, set up a Supabase database with pgvector, and optionally set up Neo4j for knowledge graph features. The server exposes MCP tools for crawling, searching, and querying crawled content.

Key features of Crawl4ai Rag

  • Smart URL detection (regular pages, sitemaps, text files)
  • Recursive crawling following internal links
  • Parallel processing for efficient multi-page crawling
  • Content chunking by headers and size
  • Vector search with optional source filtering
  • Advanced RAG strategies (contextual embeddings, hybrid search, agentic RAG, reranking, knowledge graph)

Use cases of Crawl4ai Rag

  • AI coding assistants scraping and indexing documentation for code-aware RAG
  • Building a knowledge engine for AI agents to crawl and query websites
  • Performing semantic search over previously crawled content with source filtering
  • Detecting AI hallucinations by validating code against a repository knowledge graph

常见问题

1

What dependencies does Crawl4ai Rag require?

Requires Docker or Python 3.12+, a Supabase project (for RAG), and an OpenAI API key. Neo4j is optional for knowledge graph functionality.

2

How are RAG strategies configured?

Each strategy (contextual embeddings, hybrid search, agentic RAG, reranking, knowledge graph) is enabled via a `USE_*` environment variable set to `"true"` in the `.env` file. All default to `"false"`.

3

Where is crawled content stored?

Content is stored in a Supabase database with the pgvector extension. A separate table is created for agentic RAG code examples if enabled. Knowledge graph data is stored in a Neo4j graph database.

4

What transport does the server use?

The server uses SSE (Server-Sent Events) transport by default, configurable via the `TRANSPORT` environment variable.

5

Are there known limitations or planned improvements?

The repository is a testbed for integration into Archon; known areas for improvement include supporting multiple embedding models, local Ollama execution, faster crawling/indexing, and enhanced chunking strategies.

评论

记忆与知识 分类下的更多 MCP 服务器

Codebase Memory

DeusData

High-performance code intelligence MCP server. Indexes codebases into a persistent knowledge graph — average repo in milliseconds. 162 languages, sub-ms queries, 99% fewer tokens. Single static binary, zero dependencies.

46.7K

M1K3

Round-Tower

Local AI agent for your Mac with a built-in MCP server: 18 tools for your notes and PDFs, a temporal memory graph, an on-device LLM and a neural voice. No account, no telemetry. No data centers required.

7

Sooveryn

PANDAPROG

Sooveryn gives Claude Code a team of AI personas that remembers your project. Soov, the partner and team lead, keeps your decisions and how you work in a lasting memory, and finds them again by words and by meaning, from

Leviathan Systems Public Content MCP

Leviathan Systems

Search published GPU deployment, rack integration and structured cabling resources; read canonical documents and Leviathan Systems company service scope. Anonymous, read-only Streamable HTTP MCP. No credentials required.

The Cloud

Teka

The Cloud is the AI-native workspace that gives every AI — Claude, ChatGPT, Gemini, and other Frontier & Free or Local models — shared memory and real agents. Not cloud computing: thecloud.so is one workspace where your

广告合作