Crawlbase Web MCP Server
@crawlbase
关于 Crawlbase Web MCP Server
Crawlbase MCP Server connects AI agents and LLMs with real-time web data. It powers Claude, Cursor, and Windsurf integrations with battle-tested web scraping, JavaScript rendering, and anti-bot protection enabling structured, live data inside your AI workflows.
基本信息
配置
使用下面的配置,将此服务器添加到你的 MCP 客户端。
{
"mcpServers": {
"crawlbase": {
"type": "stdio",
"command": "npx",
"args": [
"@crawlbase/mcp@latest"
],
"env": {
"CRAWLBASE_TOKEN": "your_token_here",
"CRAWLBASE_JS_TOKEN": "your_js_token_here"
}
}
}
}工具
3Crawl a URL and return HTML content
Crawl a URL and extract clean markdown content
Take a screenshot of a webpage
概览
What is Crawlbase Web MCP Server?
Crawlbase Web MCP Server is a Model Context Protocol (MCP) server that bridges AI agents with live web content. It uses Crawlbase’s crawling infrastructure, trusted by over 70,000 developers, to fetch fresh, structured, real-time data from the web. It handles JavaScript rendering, proxy rotation, anti-bot evasion, and returns HTML, Markdown, or screenshots.
How to use Crawlbase Web MCP Server?
Sign up at Crawlbase to get free Normal and JavaScript tokens. Configure the MCP server in a client like Claude Desktop, Claude Code, Cursor IDE, or Windsurf IDE by adding its JSON configuration with your tokens. Then use commands like crawl, crawl_markdown, or crawl_screenshot to fetch web data. An HTTP transport mode is also available for multi‑user or custom integrations.
Key features of Crawlbase Web MCP Server
- Fetches live web data for AI agents
- JavaScript rendering for modern web apps
- Proxy rotation and anti‑bot evasion
- Structured outputs: HTML, Markdown, screenshots
- Async crawling with Crawlbase Cloud Storage
- HTTP transport with per‑request token authentication
Use cases of Crawlbase Web MCP Server
- Market research by pulling live competitor data and reports
- E‑commerce monitoring for real‑time product, review, and price tracking
- News and finance feeds to keep AI agents current with events
- Autonomous AI agents that can act on fresh web data
FAQ from Crawlbase Web MCP Server
What do I need to get started?
You need free tokens from Crawlbase (a Normal token for HTML and a JavaScript token for JS‑rendered pages). Then configure the MCP server in your AI client.
Which AI clients are supported?
Claude Desktop, Claude Code, Cursor IDE, and Windsurf IDE are explicitly supported via stdio transport. HTTP transport mode is also available for any HTTP‑capable client.
Does it handle JavaScript‑heavy sites?
Yes. The server uses Crawlbase’s JavaScript rendering capability, which requires a JavaScript token. It handles modern web apps and anti‑bot measures.
Can I store crawl results for later use?
Yes. Crawlbase MCP can push results to Crawlbase Cloud Storage. Your AI agent can later read, list, or delete stored pages across sessions.
How does authentication work in HTTP mode?
Authentication is via environment variables or per‑request headers (X-Crawlbase-Token and X-Crawlbase-JS-Token). Headers override environment variables, enabling multi‑tenant deployments.
其他 分类下的更多 MCP 服务器
🚀 Model Context Protocol (MCP) Curriculum for Beginners
microsoftThis open-source curriculum introduces the fundamentals of Model Context Protocol (MCP) through real-world, cross-language examples in .NET, Java, TypeScript, JavaScript, Rust and Python. Designed for developers, it focuses on practical techniques for building modular, scalable,
Nginx UI
0xJackyYet another WebUI for Nginx
Activepieces
activepiecesAI Agents & MCPs & AI Workflow Automation • (~400 MCP servers for AI agents) • AI Automation / AI Agent with MCPs • AI Workflows & AI Agents • MCPs for AI Agents
Reactive Resume
amruthpillaiA one-of-a-kind resume builder that keeps your privacy in mind. Completely secure, customizable, portable, open-source and free forever. Try it out today!
Inbox Zero AI MCP
elie222The world's best AI personal assistant for email. Open source app to help you reach inbox zero fast.
评论