广告合作
MCP.so
登录
C

Crawl4ai Rag MCP Server

@coleam00

代码仓库

基本信息

分类

记忆与知识

传输方式

stdio

作者

coleam00

提交者

Tariq Forde

关于 Crawl4ai Rag MCP Server

暂无概览

#crawl4ai#web-crawling#ai-coding-assistant

Crawl4ai Rag MCP Server 配置

使用下面的配置,将此服务器添加到你的 MCP 客户端。

{
  "mcpServers": {
    "crawl4ai-rag": {
      "command": "python",
      "args": [
        "path/to/crawl4ai-mcp/src/crawl4ai_mcp.py"
      ],
      "env": {
        "TRANSPORT": "stdio",
        "OPENAI_API_KEY": "your_openai_api_key",
        "SUPABASE_URL": "your_supabase_url",
        "SUPABASE_SERVICE_KEY": "your_supabase_service_key",
        "USE_KNOWLEDGE_GRAPH": "false",
        "NEO4J_URI": "bolt://localhost:7687",
        "NEO4J_USER": "neo4j",
        "NEO4J_PASSWORD": "your_neo4j_password"
      }
    }
  }
}

Crawl4ai Rag MCP Server 工具

工具列表暂未提供

当前收录暂时没有可展示的工具列表，不代表服务没有工具。请查阅项目文档了解支持的能力。

工具是从 README 中自动提取的。维护者可以在 ## Tools 标题下列出工具,即可填充这部分内容。

概览

What is Crawl4ai Rag?

A Model Context Protocol (MCP) server that integrates Crawl4AI web crawling with Supabase vector storage to provide AI agents and coding assistants with web crawling and RAG (Retrieval-Augmented Generation) capabilities.

How to use Crawl4ai Rag?

Install via Docker or uv (Python 3.12+), configure a .env file with Supabase and OpenAI API key, set up a Supabase database with pgvector, and optionally set up Neo4j for knowledge graph features. The server exposes MCP tools for crawling, searching, and querying crawled content.

Key features of Crawl4ai Rag

  • Smart URL detection (regular pages, sitemaps, text files)
  • Recursive crawling following internal links
  • Parallel processing for efficient multi-page crawling
  • Content chunking by headers and size
  • Vector search with optional source filtering
  • Advanced RAG strategies (contextual embeddings, hybrid search, agentic RAG, reranking, knowledge graph)

Use cases of Crawl4ai Rag

  • AI coding assistants scraping and indexing documentation for code-aware RAG
  • Building a knowledge engine for AI agents to crawl and query websites
  • Performing semantic search over previously crawled content with source filtering
  • Detecting AI hallucinations by validating code against a repository knowledge graph

常见问题

1

What dependencies does Crawl4ai Rag require?

Requires Docker or Python 3.12+, a Supabase project (for RAG), and an OpenAI API key. Neo4j is optional for knowledge graph functionality.

2

How are RAG strategies configured?

Each strategy (contextual embeddings, hybrid search, agentic RAG, reranking, knowledge graph) is enabled via a `USE_*` environment variable set to `"true"` in the `.env` file. All default to `"false"`.

3

Where is crawled content stored?

Content is stored in a Supabase database with the pgvector extension. A separate table is created for agentic RAG code examples if enabled. Knowledge graph data is stored in a Neo4j graph database.

4

What transport does the server use?

The server uses SSE (Server-Sent Events) transport by default, configurable via the `TRANSPORT` environment variable.

5

Are there known limitations or planned improvements?

The repository is a testbed for integration into Archon; known areas for improvement include supporting multiple embedding models, local Ollama execution, faster crawling/indexing, and enhanced chunking strategies.

评论

记忆与知识 分类下的更多 MCP 服务器

PLUR

plur-ai

AI agents start every session with amnesia — you re-explain the project, repeat your preferences, and correct the same mistakes over and over. PLUR gives them a memory that persists. Your agent's corrections, preference

226

Agimem

Persistent memory for AI agents. Hosted MCP server with capsule-scoped key/value storage, tags, TTL expiration, bulk ops, and RBAC.

Groundwork

Rarefied-Earth

Groundwork: company memory for your AI tools. Docs public. Source closed. Public proof needs no account.

1

OpenLore

aakarim

Serve your docs to AI agents over SSH and MCP. Agents use ls, cat, grep, find, and pipes against a virtual filesystem backed by your documentation. Identity-scoped access, governed writing with human approval, and portab

66

Stele

Stele-Dev

Shared memory for AI coding agents. Claude Code, Cursor and Codex read your project's decisions, lessons and tasks before they act.

广告合作