WebScraping.AI MCP Server

Name: WebScraping.AI MCP Server
Author: MCP-Mirror

@MCP-Mirror

リポジトリ

WebScraping.AI MCP Server について

Mirror of

#webscraping #data-extraction #automation

基本情報

カテゴリ

その他

ランタイム

node

トランスポート

stdio

公開者

MCP-Mirror

設定

以下の設定を使って、このサーバーを MCP 対応クライアントに追加してください。

{
  "mcpServers": {
    "webscraping-ai_webscraping-ai-mcp-server": {
      "command": "npx",
      "args": [
        "@modelcontextprotocol/inspector",
        "node",
        "src/index.js"
      ]
    }
  }
}

ツール

ツールは検出されませんでした

ツールは README から自動的に抽出されます。メンテナーは ## Tools という見出しの下に記載することで、このタブに反映できます。

概要

What is WebScraping.AI MCP Server?

WebScraping.AI MCP Server is a Model Context Protocol (MCP) server that integrates with the WebScraping.AI API to provide web data extraction capabilities to MCP-enabled LLMs. It allows AI agents to scrape web pages, answer questions about content, extract structured data, and retrieve HTML or text with JavaScript rendering and proxy support.

How to use WebScraping.AI MCP Server?

Install and run it via npx with your API key: env WEBSCRAPING_AI_API_KEY=your_api_key npx -y webscraping-ai-mcp. Configure it in Cursor (.cursor/mcp.json) or Claude Desktop (claude_desktop_config.json) by specifying the command and environment variables. All scraping tools accept parameters like url, timeout, js, proxy, country, device, and wait_for.

Key features of WebScraping.AI MCP Server

Question answering about web page content
Structured data extraction with custom fields
Full HTML and plain text retrieval
CSS selector–based content extraction
Support for datacenter and residential proxies with country selection
JavaScript rendering using headless Chrome/Chromium

Use cases of WebScraping.AI MCP Server

Ask an LLM to summarize or find specific information on a webpage
Extract product details (title, price, description) from e-commerce sites
Retrieve the full HTML of a page for further analysis or archiving
Monitor account usage (API requests remaining) via the account tool
Scrape content from multiple page sections using multiple CSS selectors

FAQ from WebScraping.AI MCP Server

What is the difference between WebScraping.AI MCP Server and direct web scraping?

This server wraps WebScraping.AI’s API into the MCP format, making it accessible to any MCP-compatible LLM. It handles rate limiting, retries, proxy selection, and JavaScript rendering automatically, so the LLM can focus on extraction tasks.

What are the runtime dependencies?

You need Node.js and an active WebScraping.AI API key. The server runs via npx or can be installed from npm; no other external services are required.

Where does scraped data live?

All web page data is fetched through the WebScraping.AI API. The server does not store data locally; responses are returned directly to the calling LLM.

Are there any rate limits or concurrency restrictions?

Yes. The default concurrency limit is 5 requests (configurable via WEBSCRAPING_AI_CONCURRENCY_LIMIT). The server includes automatic backoff for rate-limit errors (HTTP 429). Timeouts are capped at 30 s for page retrieval and 2 s for JavaScript rendering (configurable).

What transport and authentication does the server use?

The server communicates over stdio (standard input/output) using the MCP protocol. Authentication is via an API key passed as the environment variable WEBSCRAPING_AI_API_KEY.