WebScraping.AI MCP Server

Name: WebScraping.AI MCP Server
Author: webscraping-ai

@webscraping-ai

关于 WebScraping.AI MCP Server

A Model Context Protocol (MCP) server implementation that integrates with WebScraping.AI for web data extraction capabilities.

#scraping #web #http #html #parsing #extraction #data

基本信息

分类

其他

运行时

node

传输方式

stdio

发布者

webscraping-ai

配置

使用下面的配置,将此服务器添加到你的 MCP 客户端。

{
  "mcpServers": {
    "WebScraping.AI": {
      "command": "npx",
      "args": [
        "-y",
        "webscraping-ai-mcp"
      ],
      "env": {
        "WEBSCRAPING_AI_API_KEY": "test-api-key"
      }
    }
  }
}

工具

未检测到工具

工具是从 README 中自动提取的。维护者可以在 ## Tools 标题下列出工具,即可填充这部分内容。

概览

What is WebScraping.AI MCP Server?

A Model Context Protocol (MCP) server that integrates with WebScraping.AI to provide web data extraction capabilities for LLM platforms like Claude. It enables AI agents to scrape, extract, and answer questions from web page content.

How to use WebScraping.AI MCP Server?

Run with npx -y webscraping-ai-mcp after setting the required WEBSCRAPING_AI_API_KEY environment variable. Configure in Cursor (.cursor/mcp.json) or Claude Desktop (claude_desktop_config.json) with optional settings like WEBSCRAPING_AI_CONCURRENCY_LIMIT (default 5) and WEBSCRAPING_AI_ENABLE_CONTENT_SANDBOXING. Tools are invoked via MCP tool calls.

Key features of WebScraping.AI MCP Server

Question answering about web page content
Structured data extraction from web pages
HTML content retrieval with JavaScript rendering
Plain text extraction from web pages
CSS selector-based content extraction
Multiple proxy types (datacenter, residential, stealth)
Content sandboxing to prevent prompt injection

Use cases of WebScraping.AI MCP Server

Answer specific questions from any public web page
Extract product details (title, price, description) from e‑commerce sites
Retrieve full HTML or visible text for content analysis
Scrape multiple page sections using CSS selectors
Monitor account usage and remaining API quota

FAQ from WebScraping.AI MCP Server

What configuration is required?

You must set the WEBSCRAPING_AI_API_KEY environment variable. Optional settings include concurrency limit, proxy type, JavaScript rendering, and content sandboxing.

What is content sandboxing?

When enabled (WEBSCRAPING_AI_ENABLE_CONTENT_SANDBOXING=true), scraped content is wrapped with security boundaries and a warning not to execute commands from that section, helping protect LLMs against indirect prompt injection.

What proxy types are available?

Three proxy types: datacenter, residential, and stealth. Residential is the default. Stealth is recommended for heavily protected sites but costs more.

How does the server handle errors?

It provides automatic retries for transient errors, rate limit handling with backoff, and returns detailed error messages (e.g., HTTP 429).

What transport does the server use?

The server connects via STDIO transport, configured as a command in MCP‑enabled platforms. Authentication is done through the API key passed as an environment variable.