广告合作
MCP.so
登录

WebScraping.AI MCP Server

@webscraping-ai

代码仓库主页

基本信息

分类

其他

传输方式

stdio

作者

webscraping-ai

关于 WebScraping.AI MCP Server

A Model Context Protocol (MCP) server implementation that integrates with WebScraping.AI for web data extraction capabilities.

#scraping#web#http#html#parsing#extraction#data

WebScraping.AI MCP Server 配置

使用下面的配置,将此服务器添加到你的 MCP 客户端。

{
  "mcpServers": {
    "WebScraping.AI": {
      "command": "npx",
      "args": [
        "-y",
        "webscraping-ai-mcp"
      ],
      "env": {
        "WEBSCRAPING_AI_API_KEY": "test-api-key"
      }
    }
  }
}

WebScraping.AI MCP Server 工具

7

概览

What is WebScraping.AI MCP Server?

A Model Context Protocol (MCP) server that integrates with WebScraping.AI to provide web data extraction capabilities for LLM platforms like Claude. It enables AI agents to scrape, extract, and answer questions from web page content.

How to use WebScraping.AI MCP Server?

Run with npx -y webscraping-ai-mcp after setting the required WEBSCRAPING_AI_API_KEY environment variable. Configure in Cursor (.cursor/mcp.json) or Claude Desktop (claude_desktop_config.json) with optional settings like WEBSCRAPING_AI_CONCURRENCY_LIMIT (default 5) and WEBSCRAPING_AI_ENABLE_CONTENT_SANDBOXING. Tools are invoked via MCP tool calls.

Key features of WebScraping.AI MCP Server

  • Question answering about web page content
  • Structured data extraction from web pages
  • HTML content retrieval with JavaScript rendering
  • Plain text extraction from web pages
  • CSS selector-based content extraction
  • Multiple proxy types (datacenter, residential, stealth)
  • Content sandboxing to prevent prompt injection

Use cases of WebScraping.AI MCP Server

  • Answer specific questions from any public web page
  • Extract product details (title, price, description) from e‑commerce sites
  • Retrieve full HTML or visible text for content analysis
  • Scrape multiple page sections using CSS selectors
  • Monitor account usage and remaining API quota

常见问题

1

What configuration is required?

You must set the `WEBSCRAPING_AI_API_KEY` environment variable. Optional settings include concurrency limit, proxy type, JavaScript rendering, and content sandboxing.

2

What is content sandboxing?

When enabled (`WEBSCRAPING_AI_ENABLE_CONTENT_SANDBOXING=true`), scraped content is wrapped with security boundaries and a warning not to execute commands from that section, helping protect LLMs against indirect prompt injection.

3

What proxy types are available?

Three proxy types: `datacenter`, `residential`, and `stealth`. Residential is the default. Stealth is recommended for heavily protected sites but costs more.

4

How does the server handle errors?

It provides automatic retries for transient errors, rate limit handling with backoff, and returns detailed error messages (e.g., HTTP 429).

5

What transport does the server use?

The server connects via STDIO transport, configured as a command in MCP‑enabled platforms. Authentication is done through the API key passed as an environment variable.

评论

其他 分类下的更多 MCP 服务器

Figs, Relationship Expert

Figs O'Sullivan, LMFT

Couples therapist Figs O'Sullivan (LMFT, 17 years, endorsed by Sue Johnson, creator of EFT) inside your AI. Ask relationship questions answered from his clinical writing, name the pattern in a fight, get a three-step rep

YOUB

Connect your AI assistant to your own YOUB endurance training account. Read recorded activities with full analysis, planned sessions, race goals, wellness (HRV, sleep, resting HR) and your athletic profile. Create or mov

🚀 Model Context Protocol (MCP) Curriculum for Beginners

microsoft

This open-source curriculum introduces the fundamentals of Model Context Protocol (MCP) through real-world, cross-language examples in .NET, Java, TypeScript, JavaScript, Rust and Python. Designed for developers, it focuses on practical techniques for building modular, scalable,

16.7K

Lemon.io

lemon-io

Request dedicated senior developers to work on your project, write job descriptions, and prep technical interviews — all without leaving your Claude chat. Just describe what you need, and Lemon.io MCP does the rest. 1–3

MaxKB

1Panel-dev

🔥 MaxKB is an open-source platform for building enterprise-grade agents. 强大易用的开源企业级智能体平台。

22.0K
广告合作