Fastcrw

Name: Fastcrw
Author: us

@us

关于 Fastcrw

Fast, lightweight Firecrawl/Tavily alternative in Rust. Web scraper, crawler & search API with MCP server for AI agents. Drop-in Firecrawl-compatible API (/scrape, /crawl, /search). 2.3x faster than Tavily, 1.5x faster than Firecrawl in 1K-URL benchmarks. 6 MB RAM, single binary.

#web-scraping #crawling #data-extraction #ai-agents #rag #llm #markdown #search #self-hosted #firecrawl-alternative #browser-automation #structured-data #mcp-server #claude #curso

基本信息

分类

浏览器自动化

许可证

AGPL-3.0

运行时

rust

传输方式

stdio

发布者

提交者

Recep Saritekin

配置

使用下面的配置,将此服务器添加到你的 MCP 客户端。

{
  "mcpServers": {
    "crw": {
      "command": "npx",
      "args": [
        "crw-mcp"
      ],
      "env": {
        "CRW_API_KEY": "<YOUR_API_KEY>"
      }
    }
  }
}

工具

未检测到工具

工具是从 README 中自动提取的。维护者可以在 ## Tools 标题下列出工具,即可填充这部分内容。

概览

What is Fastcrw?

Fastcrw is a self-hosted, Rust-native web crawler and scraper designed for AI agents. It serves as an open-source alternative to Firecrawl, offering a single static binary with a native REST API under /v1/* (scrape, crawl, map, search, structured extraction, and change tracking) plus a /firecrawl/v2/* compatibility layer for migrating from Firecrawl. It can be self-hosted for free under AGPL-3.0 or used via the managed API at api.fastcrw.com.

How to use Fastcrw?

Use Fastcrw by running the self-hosted binary (e.g., docker run -p 3000:3000 ghcr.io/us/crw) and sending requests to its REST endpoints (/v1/scrape, /v1/crawl, etc.) with an API key. Alternatively, install the built-in MCP server via npm install -g crw-mcp for use with MCP-compatible AI agents like Claude Code, Cursor, or Windsurf.

Key features of Fastcrw

Single static binary (~8 MB), no Redis, Node.js, or Python required.
~50 MB RAM idle, runs on a $5 VPS.
Native /v1/* API and Firecrawl v2 compatibility layer.
Change tracking with markdown git-diff and optional LLM judge.
AGPL-3.0 open core with managed commercial option.
Built-in MCP server and SDKs (npm, PyPI, Homebrew, Cargo).

Use cases of Fastcrw

AI agents scraping web content (markdown, HTML, JSON) programmatically.
Crawling multi-page documentation sites for knowledge base ingestion.
Mapping website link structures for sitemap generation or audit.
Structured JSON extraction from web pages using a JSON Schema.
Monitoring page changes with diff snapshots for alerting or tracking.

FAQ from Fastcrw

How does Fastcrw compare to Firecrawl?

Fastcrw is a Rust-native alternative that runs as a single static binary (~8 MB, ~50 MB RAM idle) compared to Firecrawl’s multi-container setup. It offers a /firecrawl/v2/* compatibility layer for easy migration and natively supports change tracking and MCP server integration.

What are the runtime dependencies?

None beyond the binary itself. There is no need for Redis, Node.js, Python, or a headless browser alongside the request path. Self-hosted search requires a SearXNG sidecar (via Docker Compose) if used.

Where does my data live?

Data stays entirely on your infrastructure when self-hosting (the binary runs locally). If using the managed API at api.fastcrw.com, data passes through their proxy network and dashboard.

What transports and authentication does Fastcrw support?

Transport is HTTP REST (TCP) with Bearer token authentication via Authorization: Bearer <key> header. The built-in MCP server uses the MCP protocol (stdio or SSE). Embedded mode runs in-process with no external transport.

Is there a usage limit for the self-hosted version?

The README does not specify hard limits. The memory baseline is ~50 MB idle, scaling with crawl queue size. The managed API offers 500 free credits on sign-up.