广告合作
MCP.so
登录

Crawlbase Web MCP Server

@crawlbase

代码仓库主页

基本信息

分类

其他

传输方式

stdio

作者

crawlbase

提交者

Muhammad Waleed

关于 Crawlbase Web MCP Server

Crawlbase MCP Server connects AI agents and LLMs with real-time web data. It powers Claude, Cursor, and Windsurf integrations with battle-tested web scraping, JavaScript rendering, and anti-bot protection enabling structured, live data inside your AI workflows.

#web-scraping#web-scraping-mcp#web-crawling-mcp#web-mcp-server#web-data-mcp#web-data#ai#agents

Crawlbase Web MCP Server 配置

使用下面的配置,将此服务器添加到你的 MCP 客户端。

{
  "mcpServers": {
    "crawlbase": {
      "type": "stdio",
      "command": "npx",
      "args": [
        "@crawlbase/mcp@latest"
      ],
      "env": {
        "CRAWLBASE_TOKEN": "your_token_here",
        "CRAWLBASE_JS_TOKEN": "your_js_token_here"
      }
    }
  }
}

Crawlbase Web MCP Server 工具

3

Crawl a URL and return HTML content

Crawl a URL and extract clean markdown content

Take a screenshot of a webpage

概览

What is Crawlbase Web MCP Server?

Crawlbase Web MCP Server is a Model Context Protocol (MCP) server that bridges AI agents with live web content. It uses Crawlbase’s crawling infrastructure, trusted by over 70,000 developers, to fetch fresh, structured, real-time data from the web. It handles JavaScript rendering, proxy rotation, anti-bot evasion, and returns HTML, Markdown, or screenshots.

How to use Crawlbase Web MCP Server?

Sign up at Crawlbase to get free Normal and JavaScript tokens. Configure the MCP server in a client like Claude Desktop, Claude Code, Cursor IDE, or Windsurf IDE by adding its JSON configuration with your tokens. Then use commands like crawl, crawl_markdown, or crawl_screenshot to fetch web data. An HTTP transport mode is also available for multi‑user or custom integrations.

Key features of Crawlbase Web MCP Server

  • Fetches live web data for AI agents
  • JavaScript rendering for modern web apps
  • Proxy rotation and anti‑bot evasion
  • Structured outputs: HTML, Markdown, screenshots
  • Async crawling with Crawlbase Cloud Storage
  • HTTP transport with per‑request token authentication

Use cases of Crawlbase Web MCP Server

  • Market research by pulling live competitor data and reports
  • E‑commerce monitoring for real‑time product, review, and price tracking
  • News and finance feeds to keep AI agents current with events
  • Autonomous AI agents that can act on fresh web data

常见问题

1

What do I need to get started?

You need free tokens from Crawlbase (a Normal token for HTML and a JavaScript token for JS‑rendered pages). Then configure the MCP server in your AI client.

2

Which AI clients are supported?

Claude Desktop, Claude Code, Cursor IDE, and Windsurf IDE are explicitly supported via stdio transport. HTTP transport mode is also available for any HTTP‑capable client.

3

Does it handle JavaScript‑heavy sites?

Yes. The server uses Crawlbase’s JavaScript rendering capability, which requires a JavaScript token. It handles modern web apps and anti‑bot measures.

4

Can I store crawl results for later use?

Yes. Crawlbase MCP can push results to Crawlbase Cloud Storage. Your AI agent can later read, list, or delete stored pages across sessions.

5

How does authentication work in HTTP mode?

Authentication is via environment variables or per‑request headers (`X-Crawlbase-Token` and `X-Crawlbase-JS-Token`). Headers override environment variables, enabling multi‑tenant deployments.

评论

其他 分类下的更多 MCP 服务器

N

Next Generation Movers: free moving quotes (remote MCP)

Next Generation Movers

Free moving quotes for AI agents from Next Generation Movers, a licensed interstate mover (USDOT 3764697). US long-distance nationwide and local DC/MD/VA. Indicative ballpark price and booking options; optional $0.01 x40

Sequential Thinking

modelcontextprotocol

Model Context Protocol Servers

88.2K

YOUB

Connect your AI assistant to your own YOUB endurance training account. Read recorded activities with full analysis, planned sessions, race goals, wellness (HRV, sleep, resting HR) and your athletic profile. Create or mov

Glasswarp

Glasswarp

See and control a real Windows PC you own — from any MCP client, locally or remotely. Observe (UIA + screenshots), click/type/drag/scroll, launch apps, owner Live View. BYOH: your machine, your key.

DeepMark

DeepMark helps teachers deliver rapid, consistent marking with meaningful feedback for every student — in a fraction of the time. What once took a week, now takes one free period.

广告合作