WebScraping.AI MCP Server

@MCP-Mirror

リポジトリ

基本情報

カテゴリ

その他

トランスポート

stdio

作者

MCP-Mirror

WebScraping.AI MCP Server について

Mirror of

#webscraping #data-extraction #automation

WebScraping.AI MCP Server の設定

以下の設定を使って、このサーバーを MCP 対応クライアントに追加してください。

{
  "mcpServers": {
    "webscraping-ai_webscraping-ai-mcp-server": {
      "command": "npx",
      "args": [
        "@modelcontextprotocol/inspector",
        "node",
        "src/index.js"
      ]
    }
  }
}

WebScraping.AI MCP Server のツール

ツール一覧はまだありません

この掲載には表示できるツール一覧がまだありません。サーバーにツールがないという意味ではありません。対応機能はプロジェクトのドキュメントをご確認ください。

上のボタンで一時的なサンドボックスでこのサーバーを実行し、ツール一覧を取得できます。

概要

What is WebScraping.AI MCP Server?

WebScraping.AI MCP Server is a Model Context Protocol (MCP) server that integrates with the WebScraping.AI API to provide web data extraction capabilities to MCP-enabled LLMs. It allows AI agents to scrape web pages, answer questions about content, extract structured data, and retrieve HTML or text with JavaScript rendering and proxy support.

How to use WebScraping.AI MCP Server?

Install and run it via npx with your API key: env WEBSCRAPING_AI_API_KEY=your_api_key npx -y webscraping-ai-mcp. Configure it in Cursor (.cursor/mcp.json) or Claude Desktop (claude_desktop_config.json) by specifying the command and environment variables. All scraping tools accept parameters like url, timeout, js, proxy, country, device, and wait_for.

Key features of WebScraping.AI MCP Server

Question answering about web page content
Structured data extraction with custom fields
Full HTML and plain text retrieval
CSS selector–based content extraction
Support for datacenter and residential proxies with country selection
JavaScript rendering using headless Chrome/Chromium

Use cases of WebScraping.AI MCP Server

Ask an LLM to summarize or find specific information on a webpage
Extract product details (title, price, description) from e-commerce sites
Retrieve the full HTML of a page for further analysis or archiving
Monitor account usage (API requests remaining) via the account tool
Scrape content from multiple page sections using multiple CSS selectors

よくある質問

What is the difference between WebScraping.AI MCP Server and direct web scraping?

This server wraps WebScraping.AI’s API into the MCP format, making it accessible to any MCP-compatible LLM. It handles rate limiting, retries, proxy selection, and JavaScript rendering automatically, so the LLM can focus on extraction tasks.

What are the runtime dependencies?

You need Node.js and an active WebScraping.AI API key. The server runs via `npx` or can be installed from npm; no other external services are required.

Where does scraped data live?

All web page data is fetched through the WebScraping.AI API. The server does not store data locally; responses are returned directly to the calling LLM.

Are there any rate limits or concurrency restrictions?

Yes. The default concurrency limit is 5 requests (configurable via `WEBSCRAPING_AI_CONCURRENCY_LIMIT`). The server includes automatic backoff for rate-limit errors (HTTP 429). Timeouts are capped at 30 s for page retrieval and 2 s for JavaScript rendering (configurable).

What transport and authentication does the server use?

The server communicates over stdio (standard input/output) using the MCP protocol. Authentication is via an API key passed as the environment variable `WEBSCRAPING_AI_API_KEY`.

「その他」の他のコンテンツ

Maestro

mobile-dev-inc

Painless E2E Automation for Mobile and Web

14.8K

Handl

Handl speaks the Model Context Protocol. Your whole billing operation — projects, invoices, clients, reports, approvals — as tools any AI assistant can call, inside your autonomy rules.