Web Scraping with Anthropic’s MCP Server

@luminati-io

代码仓库

基本信息

分类

浏览器自动化

传输方式

stdio

作者

luminati-io

关于 Web Scraping with Anthropic’s MCP Server

Example MCP server and instructions for connecting Anthropic LLMs to external web scraping tools, with real-world examples and Bright Data integration.

#mcp #web-scraping #anthropic-claude#scraping-mcp

Web Scraping with Anthropic’s MCP Server 配置

使用下面的配置,将此服务器添加到你的 MCP 客户端。

{
  "mcpServers": {
    "web-scraping-with-mcp": {
      "command": "python",
      "args": [
        "-m",
        "venv",
        "mcp-amazon-scraper"
      ]
    }
  }
}

Web Scraping with Anthropic’s MCP Server 工具

工具列表暂未提供

当前收录暂时没有可展示的工具列表，不代表服务没有工具。请查阅项目文档了解支持的能力。

工具是从 README 中自动提取的。维护者可以在 ## Tools 标题下列出工具,即可填充这部分内容。

概览

What is Web Scraping with Anthropic’s MCP?

Web Scraping with Anthropic’s MCP is a guide that explains how to set up an MCP server for on-demand data extraction, connect it with development tools like Claude Desktop or Cursor, and optionally leverage Bright Data for professional web information. It is intended for developers who want to enable LLMs to interact with real-world websites in a standardized, secure way.

How to use Web Scraping with Anthropic’s MCP?

The guide walks through creating a Python MCP server using the MCP Python SDK, Playwright, and lxml. You install dependencies (mcp, playwright, lxml), write a server file that exposes tools like fetch_page and extract_info, then connect the server to an MCP host (e.g., Claude Desktop or Cursor IDE) via stdio transport. The server can then be invoked by natural‑language prompts such as “Fetch product info from this Amazon link.”

Key features of Web Scraping with Anthropic’s MCP

Standardizes LLM interaction with external tools via JSON‑RPC
Enables real‑time web data extraction without custom integrations
Uses Playwright to render JavaScript‑heavy pages
Exposes two tools: fetch_page (download HTML) and extract_info (parse structured data)
Works with multiple MCP hosts (Claude Desktop, Cursor, etc.)

Use cases of Web Scraping with Anthropic’s MCP

Extracting structured Amazon product details (price, title, specs) via an LLM
Providing LLMs with up‑to‑date information from live websites
Building a custom scraper that integrates with AI assistants for on‑demand research
Enabling automated web data retrieval in development workflows (e.g., IDE plugins)

常见问题

Why can’t LLMs browse the web without MCP?

LLMs lack built‑in capabilities to access local files, execute scripts, or retrieve current website data. MCP bridges this gap by letting them call external tools.

What is the Model Context Protocol (MCP)?

MCP is an open standard by Anthropic that defines a uniform interface for LLMs to interact with external tools and data sources using JSON‑RPC 2.0 over transports like stdio.

What dependencies does the custom MCP server require?

Python 3, the MCP Python SDK (`mcp`), Playwright (`playwright`), and lxml (`lxml`). Playwright browser binaries must also be installed.

What transport does the server use?

The server communicates via stdio (standard input/output) as shown in the example configuration for Claude Desktop.

Does the server require authentication or API keys?

The README does not mention any authentication. The custom server runs locally and uses Playwright’s headless browser; no external API keys are required.

浏览器自动化分类下的更多 MCP 服务器

browser-use-mcp-server

co-browser

Browse the web, directly from Cursor etc.

827

Fetch

modelcontextprotocol

Model Context Protocol Servers

88.1K

TinyFish

tinyfish

Give your AI agent the live web. One MCP server for Search, Fetch, Browser, and Agent: browse real sites, extract structured data, fill forms, and run multi-step workflows. Search and Fetch are free.

Screenshot Scout

screenshotscout

Capture screenshots of webpages as images or PDFs with Screenshot Scout.

Gologin MCP Server

gologinapp

MCP server for managing Gologin browser profiles, proxies, fingerprints and cloud browser sessions through AI conversations