Web Scraping with Anthropic’s MCP Server

@luminati-io

代码仓库主页

基本信息

分类

浏览器自动化

传输方式

stdio

作者

luminati-io

关于 Web Scraping with Anthropic’s MCP Server

Example MCP server and instructions for connecting Anthropic LLMs to external web scraping tools, with real-world examples and Bright Data integration.

#mcp #web-scraping #anthropic-claude

Web Scraping with Anthropic’s MCP Server 配置

使用下面的配置,将此服务器添加到你的 MCP 客户端。

{
  "mcpServers": {
    "web-scraping-mcp": {
      "command": "python",
      "args": [
        "-m",
        "venv",
        "mcp-amazon-scraper"
      ]
    }
  }
}

Web Scraping with Anthropic’s MCP Server 工具

工具列表暂未提供

当前收录暂时没有可展示的工具列表，不代表服务没有工具。请查阅项目文档了解支持的能力。

工具是从 README 中自动提取的。维护者可以在 ## Tools 标题下列出工具,即可填充这部分内容。

概览

What is Web Scraping with Anthropic’s MCP?

This guide explains how to build and run an MCP (Model Context Protocol) server for on-demand web data extraction, enabling LLMs like Claude to retrieve live information from websites. It uses Python, Playwright, and lxml to create a server that scrapes Amazon product pages, and it covers connecting the server to development tools like Cursor or Claude Desktop. It also introduces Bright Data’s MCP for professional-grade extraction.

How to use Web Scraping with Anthropic’s MCP?

Set up a Python 3 virtual environment, install dependencies (pip install mcp playwright lxml, then python -m playwright install), and create a file amazon_scraper_mcp.py using the provided code. Run the server, then connect it via an MCP client in Cursor or Claude Desktop. The server exposes two tools: fetch_page (downloads HTML) and extract_info (parses structured data from saved HTML).

Key features of Web Scraping with Anthropic’s MCP

fetch_page tool downloads HTML via Playwright headless browser.
extract_info tool parses product data using lxml XPath selectors.
Handles JavaScript-heavy pages with configurable delays.
Saves HTML to a temporary file for structured extraction.
Designed for Amazon product pages but adaptable to any URL.

Use cases of Web Scraping with Anthropic’s MCP

Extract real-time product details (price, title, description) from Amazon.
Enable LLMs to answer questions based on current web data.
Automate data collection for price monitoring or comparison sites.
Integrate live web scraping into AI assistants and IDEs.

常见问题

What is MCP?

Model Context Protocol (MCP) is an open standard by Anthropic that allows LLMs to interact with external tools, APIs, and data sources in a consistent, secure way. It uses a JSON-RPC interface over transports like `stdio`.

Why does an LLM need an MCP server for web scraping?

LLMs cannot naturally browse the internet or fetch live data. MCP provides a standardized bridge that lets them call tools to download and extract information from websites on demand.

What dependencies are required?

Python 3, the MCP Python SDK (`mcp`), Playwright for browser automation, and lxml for HTML parsing. Playwright browser binaries must be installed separately.

What tools does this server expose?

浏览器自动化分类下的更多 MCP 服务器

Browser Control MCP

eyalzh

MCP server paired with a browser extension that enables AI agents to control the user's browser.

299

Open Browser Use

iFurySt

🔮 Platform-neutral Browser Use for AI agents: real Chrome automation with a CLI + SDKs, no lock-in, dead simple.

371

Puppeteer MCP Server

merajmehrabi

This MCP server provides browser automation capabilities through Puppeteer, allowing interaction with both new browser instances and existing Chrome windows.

474