How do I get an API key?
Visit [AnyCrawl](https://anycrawl.dev), sign up or log in, then go to the Dashboard → API Keys → Copy your key. Free sign-up gives 1,500 credits.
Anycrawl - Turn Any Website Into Llm Ready MCP Server
@any4ai
基本情報
Anycrawl - Turn Any Website Into Llm Ready MCP Server について
AnyCrawl MCP Server, with Scrape, Crawl and SERP.
Anycrawl - Turn Any Website Into Llm Ready MCP Server の設定
以下の設定を使って、このサーバーを MCP 対応クライアントに追加してください。
{
"mcpServers": {
"anycrawl-mcp": {
"command": "npx",
"args": [
"-y",
"anycrawl-mcp-server"
],
"env": {
"ANYCRAWL_API_KEY": "<YOUR_TOKEN>",
"ANYCRAWL_BASE_URL": "https://api.anycrawl.dev",
"LOG_LEVEL": "info"
}
}
}
}Anycrawl - Turn Any Website Into Llm Ready MCP Server のツール
6Scrape a single URL and extract content in selected formats. Best for: One known page (articles, docs, product pages). Not recommended for: Multi-page coverage (use anycrawl_crawl) or open-ended discovery (use anycrawl_search). RECOMMENDED: Use 'playwright' engine for best results with dynamic content and modern websites. Usage (parameters): - url: HTTP/HTTPS URL to scrape (string, required) - engine: 'playwright' | 'cheerio' | 'puppeteer' (required, default: 'playwright') - proxy: Proxy URL (string, optional) - formats: Output formats ['markdown'|'html'|'text'|'screenshot'|'screenshot@fullPage'|'rawHtml'|'json'] (optional) - timeout: Request timeout in ms (number, optional) - retry: Enable auto-retry on failure (boolean, optional) - wait_for: Wait in ms for dynamic pages (number, optional) - include_tags: HTML tags to include (string[], optional) - exclude_tags: HTML tags to exclude (string[], optional) - json_options: { schema?, user_prompt?, schema_name?, schema_description? } (optional) - extract_source: 'html' | 'markdown' (optional) Returns: { url, status, jobId?, title?, html?, markdown?, metadata?, timestamp? } Examples: - Recommended: { "url": "https://example.com", "engine": "playwright" } - With JSON extraction: { "url": "https://news.ycombinator.com", "engine": "playwright", "formats": ["markdown"], "json_options": { "user_prompt": "Extract titles", "schema_name": "Articles" } } - With JSON schema extraction: { "url": "https://example.com/article", "engine": "playwright", "json_options": { "schema_name": "Article", "schema_description": "Extract article metadata and content", "schema": { "type": "object", "properties": { "title": { "type": "string" }, "author": { "type": "string" }, "date": { "type": "string" }, "content": { "type": "string" } }, "required": ["title", "content"] } } }
Crawl an entire website with configurable depth and limits. Best for: Multi-page coverage, site mapping, content discovery. Not recommended for: Single pages (use anycrawl_scrape) or open-ended discovery (use anycrawl_search). RECOMMENDED: Use 'playwright' engine for best results with dynamic content and modern websites. Usage (parameters): - url: Starting URL to crawl (string, required) - engine: 'playwright' | 'cheerio' | 'puppeteer' (required, default: 'playwright') - max_depth: Maximum crawl depth (number, optional, default: 10) - limit: Maximum pages to crawl (number, optional, default: 100) - strategy: Crawl strategy 'all' | 'same-domain' | 'same-hostname' | 'same-origin' (optional, default: 'same-domain') - include_paths: Path patterns to include (string[], optional) - exclude_paths: Path patterns to exclude (string[], optional) - retry: Enable auto-retry on failure (boolean, optional) - poll_seconds: Polling interval for job status (number, optional) - poll_interval_ms: Polling interval in milliseconds (number, optional) - timeout_ms: Job timeout in milliseconds (number, optional) - scrape_options: Nested scrape options for each page (object, optional) Returns: { job_id, status, message } for async jobs Examples: - Recommended: { "url": "https://example.com", "engine": "playwright", "limit": 50 } - Deep crawl: { "url": "https://docs.example.com", "engine": "playwright", "max_depth": 5, "limit": 200 } - Filtered crawl: { "url": "https://blog.example.com", "engine": "playwright", "include_paths": ["/posts/*"], "exclude_paths": ["/admin/*"] }
Search the web and optionally scrape results. Best for: Open-ended discovery, finding relevant content. Not recommended for: Known URLs (use anycrawl_scrape) or comprehensive site coverage (use anycrawl_crawl). RECOMMENDED: Use limit=5 for balanced performance and cost. Use 'playwright' engine for scraping results. Usage (parameters): - query: Search query string (string, required) - engine: Search engine 'google' (optional, default: 'google') - limit: Number of results to return (number, optional, default: 5) - offset: Number of results to skip (number, optional, default: 0) - pages: Number of search result pages to process (number, optional) - lang: Language code (string, optional) - country: Country code (string, optional) - safeSearch: Safe search level 0-2 (number, optional) - scrape_options: Options for scraping search results (object, optional) Returns: Array of search results with optional scraped content Examples: - Recommended: { "query": "artificial intelligence news", "limit": 5 } - With scraping: { "query": "TypeScript tutorials", "limit": 5, "scrape_options": { "formats": ["markdown"], "engine": "playwright" } } - Localized search: { "query": "machine learning", "lang": "es", "country": "ES", "limit": 5 }
Get the status of a crawl job. Check progress, completion status, and statistics for an ongoing or completed crawl job. Usage (parameters): - job_id: The crawl job ID (string, required) Returns: { job_id, status, start_time, expires_at, credits_used, total, completed, failed } Examples: - Check status: { "job_id": "crawl_12345" }
Get the results of a completed crawl job. Retrieve the scraped content and metadata from a completed crawl job. Usage (parameters): - job_id: The crawl job ID (string, required) - skip: Number of results to skip (number, optional, default: 0) Returns: { status, total, completed, creditsUsed, next?, data[] } Examples: - Get all results: { "job_id": "crawl_12345" } - Paginated results: { "job_id": "crawl_12345", "skip": 50 }
Cancel a running crawl job. Stop an ongoing crawl job and prevent further processing. Usage (parameters): - job_id: The crawl job ID to cancel (string, required) Returns: { success: boolean, message: string } Examples: - Cancel job: { "job_id": "crawl_12345" }
概要
What is Anycrawl?
Anycrawl is an MCP server that turns any website into LLM-ready data through web scraping and crawling. It integrates with Cursor, Claude, and other MCP clients to extract content from single URLs or entire websites, supporting multiple output formats and scraping engines. It is designed for developers and AI users who need structured web data for LLM consumption.
How to use Anycrawl?
Install and run via npx with your API key: ANYCRAWL_API_KEY=YOUR-API-KEY npx -y anycrawl-mcp. For cloud usage, set the API key only; for self-hosted, additionally set ANYCRAWL_BASE_URL. The server runs in STDIO mode by default; switch to MCP (HTTP) or SSE mode by setting ANYCRAWL_MODE. Then configure your MCP client (Cursor, Claude Desktop, etc.) to connect to Anycrawl using the provided tool definitions.
Key features of Anycrawl
- Scrape single URLs with multiple output formats
- Crawl entire websites with configurable depth and limits
- Search engine integration for web search and scraping
- Multiple engines: auto, Playwright, Cheerio, Puppeteer
- Flexible output: Markdown, HTML, text, screenshots, JSON
- Async operations with status monitoring for crawl jobs
Use cases of Anycrawl
- Extracting structured content from a single page for LLM ingestion
- Collecting data from multiple related pages (e.g., blog posts, documentation)
- Gathering web content to build retrieval-augmented generation (RAG) datasets
- Monitoring website changes via periodic scraping
- Integrating web data into AI workflows in tools like Cursor or Claude
よくある質問
Can I use Anycrawl without hosting my own server?
Yes. The cloud service at `mcp.anycrawl.dev` is recommended — no server setup required. Just set your API key and use the provided endpoints.
What scraping engines are available?
Four engines: `auto` (intelligent selection), `playwright`, `cheerio`, and `puppeteer`. You can specify one per scrape or crawl.
What output formats does Anycrawl support?
Markdown, HTML, text, screenshot (full-page optional), raw HTML, and structured JSON. Multiple formats can be requested in a single request.
What transport modes does Anycrawl support?
STDIO (default), MCP over HTTP (streamable HTTP), and SSE (Server-Sent Events). Mode is selected via the `ANYCRAWL_MODE` environment variable.
「AI とエージェント」の他のコンテンツ
scvd.store
seancrecordscvd.store is an evidence observatory for agentic commerce. Independent signed observation of what other people's endpoints, artifacts and payments actually did among other things. USDC on Base, Arbitrum One, World Chain
2
Heterogent
Hire AI agents and lock USDC in trustless on-chain escrow on Solana all from your AI assistant. Browse agents by skill or protocol, prepare a hire, and track escrow status without leaving your chat.
Hostinger
hostingerModel Context Protocol (MCP) server for Hostinger API. If you don't want to install or run anything locally, connect directly to Hostinger's hosted MCP server: This opens a browser window to authorize via OAuth. Once app
148
F
Factanker
FACTANKER serves 1.18B+ evidence-backed facts about US companies, banks, nonprofits and federal spending, sourced from SEC EDGAR, FFIEC call reports, IRS 990 filings and USAspending. Every value carries its evidence: fil
Rokha
Rokha AI LLCSearch a registry of agent skills and MCP servers, run them for real (sandbox or remote MCP, no install, a trace on every run), and hire proven agents for USDC.
コメント