What is the MCP protocol and how does it work?
The MCP (Model Context Protocol) server operates using direct STDIO communication, meaning no HTTP server is required. It integrates natively with Claude Desktop for automatic lifecycle management.
W
Web Scraper MCP Server
@navin4078
基本情報
Web Scraper MCP Server について
Scrape websites and let them talk to your LLM
Web Scraper MCP Server の設定
以下の設定を使って、このサーバーを MCP 対応クライアントに追加してください。
{
"mcpServers": {
"web-scraper": {
"command": "/full/path/to/your/venv/bin/python",
"args": [
"/full/path/to/your/app_mcp.py"
]
}
}
}Web Scraper MCP Server のツール
ツール一覧はまだありません
この掲載には表示できるツール一覧がまだありません。サーバーにツールがないという意味ではありません。対応機能はプロジェクトのドキュメントをご確認ください。
ツールは README から自動的に抽出されます。メンテナーは ## Tools という見出しの下に記載することで、このタブに反映できます。
概要
What is Web Scraper MCP?
A lightweight and efficient web scraping MCP server using the direct STDIO protocol, designed for integration with Claude Desktop. It enables users to extract text, links, images, tables, metadata, and headlines from any webpage using natural language commands.
How to use Web Scraper MCP?
Install via the automated setup script (setup.sh) or manually with pip install -r requirements.txt in a Python virtual environment. Configure Claude Desktop by adding the server to claude_desktop_config.json with absolute paths to the Python interpreter and app_mcp.py. Once configured, invoke scraping tools like scrape_website, extract_headlines, extract_metadata, or get_page_info through natural language prompts in Claude.
Key features of Web Scraper MCP
- Text extraction with CSS selectors
- Link, image, and table data extraction
- Comprehensive metadata and Open Graph tags
- Headline extraction with hierarchy
- Custom CSS selector support and result limits
- Native Claude Desktop integration (STDIO)
Use cases of Web Scraper MCP
- Scrape text content from a blog or news article
- Extract all links from a link aggregator like Hacker News
- Get headlines and structure from a news website
- Retrieve images with alt text from a gallery page
- Extract metadata and Open Graph tags from any URL
よくある質問
What dependencies does Web Scraper MCP require?
The minimal dependencies are: `requests`, `beautifulsoup4`, `lxml`, and the `mcp` library. All are installed via `pip install -r requirements.txt`.
How do I configure Web Scraper MCP with Claude Desktop?
Add a new entry under `mcpServers` in your `claude_desktop_config.json` file (located in `~/Library/Application Support/Claude/` on macOS, `%APPDATA%\Claude\` on Windows, or `~/.config/Claude/` on Linux). The entry must contain the absolute path to the Python interpreter in your virtual environment and the absolute path to `app_mcp.py` as arguments.
What should I do if the MCP server is not appearing in Claude Desktop?
Verify that the file paths in your configuration are absolute and correct. Check that the virtual environment is activated and dependencies are installed. Test the server manually by running `python app_mcp.py` from the project directory. Ensure your JSON configuration is valid (no trailing commas).
What security measures are included?
The server enforces a 10-second request timeout, validates URLs and parameters, uses modern browser User-Agent headers, and includes configurable result limits to prevent overload. Users are reminded to respect `robots.txt` and practice rate limiting.
「ブラウザ自動化」の他のコンテンツ
Fetcher MCP
jae-jaeMCP server for fetch web page content using Playwright headless browser.
1.1K
BrowserTools MCP
AgentDeskAIMonitor browser logs directly from Cursor and other MCP compatible IDEs.
7.3K
C
Chrome DevTools for agents
ChromeDevToolsOfficial Chrome DevTools MCP server from Google that lets coding agents control, debug and profile a live Chrome browser.
53.6K
Scrapling Fetch MCP
cyberchittaHelps AI assistants access text content from bot-protected websites. MCP server that fetches HTML/markdown from sites with anti-automation measures using Scrapling.
101
Full Self Browsing
Lakshman TurlapatiDOM-first AI browser automation for Chrome and MCP clients.
194
コメント