Is this server ready for production use?
No. The README explicitly states the server is under development and not ready for production use.
@BjornMelin
Basic information
π High-performance MCP Server for Crawl4AI - Enable AI assistants to access web scraping, crawling, and deep research via Model Context Protocol. Faster and more efficient than FireCrawl!
No standard config provided
This server doesn't expose a parseable MCP config block in its README. See the repository for install instructions.
RepositoryCrawl web pages from a starting URL
Retrieve crawl data by ID
List all crawls or filter by domain
Search indexed documents by query
Extract structured content from a URL
Crawl4AI MCP Server is an open-source implementation of the Model Context Protocol (MCP) that integrates with the Crawl4AI web scraping and crawling library. It is deployed as a remote MCP server on CloudFlare Workers, enabling AI assistants such as Claude to perform web scraping, crawling, and structured data extraction.
Install Node.js (v18+), npm, and Wrangler, then clone the repository, run npm install, create a CloudFlare KV namespace, update wrangler.toml, and deploy with npm run deploy. Connect MCP clients (e.g., Claude Desktop) using the assigned CloudFlare Workers URL. Available tools include crawl, getCrawl, listCrawls, search, and extract.
No. The README explicitly states the server is under development and not ready for production use.
You need Node.js v18+, npm, the Wrangler CLI, and a CloudFlare account. A CloudFlare KV namespace is required for storing crawl data.
Data is stored in a CloudFlare KV namespace configured in `wrangler.toml` under the binding `CRAWL_DATA`.
The server supports both OAuth authentication (via workers-oauth-provider) and API key authentication using Bearer tokens.
The server implements the standard Model Context Protocol and can be used with any MCP client, such as Claude Desktop.
Playwright MCP server
BitBrowser MCP simple to use, no complicated procedures required. Supports a variety of intelligent agent services.
Fast, lightweight Firecrawl/Tavily alternative in Rust. Web scraper, crawler & search API with MCP server for AI agents. Drop-in Firecrawl-compatible API (/scrape, /crawl, /search). 2.3x faster than Tavily, 1.5x faster than Firecrawl in 1K-URL benchmarks. 6 MB RAM, single binary.

Model Context Protocol Servers
Give your AI agent the live web. One MCP server for Search, Fetch, Browser, and Agent: browse real sites, extract structured data, fill forms, and run multi-step workflows. Search and Fetch are free.
Comments