Fastcrw

Name: Fastcrw
Author: us

@us

Fastcrw について

Fast, lightweight Firecrawl/Tavily alternative in Rust. Web scraper, crawler & search API with MCP server for AI agents. Drop-in Firecrawl-compatible API (/scrape, /crawl, /search). 2.3x faster than Tavily, 1.5x faster than Firecrawl in 1K-URL benchmarks. 6 MB RAM, single binary.

#web-scraping #crawling #data-extraction #ai-agents #rag #llm #markdown #search #self-hosted #firecrawl-alternative #browser-automation #structured-data #mcp-server #claude #curso

基本情報

カテゴリ

ブラウザ自動化

ライセンス

AGPL-3.0

ランタイム

rust

トランスポート

stdio

公開者

投稿者

Recep Saritekin

設定

以下の設定を使って、このサーバーを MCP 対応クライアントに追加してください。

{
  "mcpServers": {
    "crw": {
      "command": "npx",
      "args": [
        "crw-mcp"
      ],
      "env": {
        "CRW_API_KEY": "<YOUR_API_KEY>"
      }
    }
  }
}

ツール

ツールは検出されませんでした

ツールは README から自動的に抽出されます。メンテナーは ## Tools という見出しの下に記載することで、このタブに反映できます。

概要

What is Fastcrw?

Fastcrw is a self-hosted, Rust-native web crawler and scraper designed for AI agents. It serves as an open-source alternative to Firecrawl, offering a single static binary with a native REST API under /v1/* (scrape, crawl, map, search, structured extraction, and change tracking) plus a /firecrawl/v2/* compatibility layer for migrating from Firecrawl. It can be self-hosted for free under AGPL-3.0 or used via the managed API at api.fastcrw.com.

How to use Fastcrw?

Use Fastcrw by running the self-hosted binary (e.g., docker run -p 3000:3000 ghcr.io/us/crw) and sending requests to its REST endpoints (/v1/scrape, /v1/crawl, etc.) with an API key. Alternatively, install the built-in MCP server via npm install -g crw-mcp for use with MCP-compatible AI agents like Claude Code, Cursor, or Windsurf.

Key features of Fastcrw

Single static binary (~8 MB), no Redis, Node.js, or Python required.
~50 MB RAM idle, runs on a $5 VPS.
Native /v1/* API and Firecrawl v2 compatibility layer.
Change tracking with markdown git-diff and optional LLM judge.
AGPL-3.0 open core with managed commercial option.
Built-in MCP server and SDKs (npm, PyPI, Homebrew, Cargo).

Use cases of Fastcrw

AI agents scraping web content (markdown, HTML, JSON) programmatically.
Crawling multi-page documentation sites for knowledge base ingestion.
Mapping website link structures for sitemap generation or audit.
Structured JSON extraction from web pages using a JSON Schema.
Monitoring page changes with diff snapshots for alerting or tracking.

FAQ from Fastcrw

How does Fastcrw compare to Firecrawl?

Fastcrw is a Rust-native alternative that runs as a single static binary (~8 MB, ~50 MB RAM idle) compared to Firecrawl’s multi-container setup. It offers a /firecrawl/v2/* compatibility layer for easy migration and natively supports change tracking and MCP server integration.

What are the runtime dependencies?

None beyond the binary itself. There is no need for Redis, Node.js, Python, or a headless browser alongside the request path. Self-hosted search requires a SearXNG sidecar (via Docker Compose) if used.

Where does my data live?

Data stays entirely on your infrastructure when self-hosting (the binary runs locally). If using the managed API at api.fastcrw.com, data passes through their proxy network and dashboard.

What transports and authentication does Fastcrw support?

Transport is HTTP REST (TCP) with Bearer token authentication via Authorization: Bearer <key> header. The built-in MCP server uses the MCP protocol (stdio or SSE). Embedded mode runs in-process with no external transport.

Is there a usage limit for the self-hosted version?

The README does not specify hard limits. The memory baseline is ~50 MB idle, scaling with crawl queue size. The managed API offers 500 free credits on sign-up.

「ブラウザ自動化」の他のコンテンツ

Scrapling Fetch MCP

cyberchitta

Helps AI assistants access text content from bot-protected websites. MCP server that fetches HTML/markdown from sites with anti-automation measures using Scrapling.

101