Claude Web Scraper MCP

Name: Claude Web Scraper MCP
Author: vishwajeetdabholkar

@vishwajeetdabholkar

Claude Web Scraper MCP について

A simple MCP server that integrates eGet web scraper with Claude for Desktop. This connector allows Claude to scrape web content through your local eGet API, enabling search, summarization, and analysis of websites directly in conversations.

#web-scraper #mcp #claude

基本情報

カテゴリ

AI とエージェント

ランタイム

python

トランスポート

stdio

公開者

vishwajeetdabholkar

設定

以下の設定を使って、このサーバーを MCP 対応クライアントに追加してください。

{
  "mcpServers": {
    "eget_mcp": {
      "command": "uv",
      "args": [
        "venv"
      ]
    }
  }
}

ツール

ツールは検出されませんでした

ツールは README から自動的に抽出されます。メンテナーは ## Tools という見出しの下に記載することで、このタブに反映できます。

概要

What is Claude Web Scraper MCP?

Claude Web Scraper MCP is a Model Context Protocol server that connects Claude for Desktop to a locally running eGet web scraper. It allows Claude to scrape website content through your local API. Designed for users who want Claude to retrieve and analyze web pages.

How to use Claude Web Scraper MCP?

First, ensure the eGet web scraper is running locally (typically at http://localhost:8000/api/v1/scrape). Next, set up the MCP server using UV, install dependencies, and create the scrape_mcp_server.py script. Then, configure Claude for Desktop by adding the server entry in claude_desktop_config.json, pointing to the virtual environment’s Python interpreter and the script path. Restart Claude for Desktop. Once configured, you can use natural language commands like “Scrape the content from https://example.com and summarize it.”

Key features of Claude Web Scraper MCP

Connects Claude for Desktop to a local eGet web scraper.
Uses a local API for web scraping (default http://localhost:8000).
Simple setup with Python, UV, and MCP SDK.
Integrates seamlessly with Claude for Desktop via JSON configuration.
Accepts natural language scrape commands from the user.
Runs entirely on your local machine—no external service needed.

Use cases of Claude Web Scraper MCP

Ask Claude to scrape and summarize any public web page.
Extract specific information from websites (e.g., headlines, prices).
Have Claude analyze scraped content for research or automation.

FAQ from Claude Web Scraper MCP

What is the eGet web scraper and do I need it?

eGet is a separate web scraper tool (from github.com/vishwajeetdabholkar/eGet-Crawler-for-ai) that must be running locally for this MCP server to work. It handles the actual page fetching and returns data via a REST API.

What are the prerequisites?

Claude for Desktop, Python 3.7+, and the eGet web scraper running on your machine. You also need to install the mcp[cli] and httpx Python packages.

How do I configure Claude for Desktop to use this server?

Edit (or create) ~/Library/Application Support/Claude/claude_desktop_config.json (on macOS) and add an entry under mcpServers with the absolute path to your virtual environment’s Python and the script file. Then restart Claude for Desktop.

Why might scraping fail?

Common issues: the eGet scraper is not running, the API endpoint in the script doesn’t match your eGet configuration, Claude is using the wrong Python interpreter, or you need to restart Claude after making config changes.

Where does scraped data live and is authentication required?

Data is fetched from the web through your local eGet instance and returned directly to Claude. No external authentication is needed for the MCP server itself—only the eGet service must be accessible locally.