Webfetch Mcp

Name: Webfetch Mcp
Author: simonediroma

@simonediroma

リポジトリ

Webfetch Mcp について

概要はまだありません

#webfetch #http-client #trafilatura #http-request #custom-http-header

基本情報

カテゴリ

その他

トランスポート

stdio

公開者

simonediroma

投稿者

simonediroma

設定

以下の設定を使って、このサーバーを MCP 対応クライアントに追加してください。

{
  "mcpServers": {
    "webfetch": {
      "command": "/absolute/path/to/.venv/bin/python",
      "args": [
        "/absolute/path/to/server.py"
      ],
      "env": {
        "WEBFETCH_CONFIG": "/absolute/path/to/webfetch.yaml"
      }
    }
  }
}

ツール

ツールは検出されませんでした

ツールは README から自動的に抽出されます。メンテナーは ## Tools という見出しの下に記載することで、このタブに反映できます。

概要

What is webfetch-mcp?

webfetch-mcp is a local Python MCP server that replaces an AI assistant’s built-in WebFetch tool with a fully configurable HTTP client. It supports domain-scoped headers, retries, proxies, timeouts, output formats, bot-block detection, and prompt-injection sanitization.

How to use webfetch-mcp?

Clone the repository, create a Python 3.10+ virtual environment, install dependencies from requirements.txt, copy and edit the YAML configuration file (webfetch.yaml.example → webfetch.yaml), then register the server’s command, args, and env in your AI assistant’s MCP server settings (e.g., ~/.claude/settings.json). Restart the assistant; the tool is exposed as mcp__webfetch__fetch.

Key features of webfetch-mcp

Domain-scoped custom headers and auth tokens.
Configurable YAML config file.
Retry with exponential backoff on failures.
Bot-block detection with optional retry.
Prompt-injection sanitization (flag or strip).
JavaScript rendering via Playwright (optional).

Use cases of webfetch-mcp

Fetching authenticated API content with per-domain tokens.
Scraping bot-protected websites (Cloudflare, Akamai).
Extracting main article content with trafilatura format.
Running CI/CD smoke tests using response assertions.
Rendering JavaScript-heavy SPAs before content extraction.

FAQ from webfetch-mcp

How is webfetch-mcp different from a built-in WebFetch tool?

The built-in tool sends requests without custom headers, so it is blocked by bot-protection systems and cannot authenticate against APIs. webfetch-mcp allows per-domain headers, retries, proxies, and output formatting, all configured automatically.

What are the runtime dependencies of webfetch-mcp?

Python 3.10+ and the packages in requirements.txt: mcp[cli], httpx, python-dotenv, markdownify, trafilatura, pyyaml, and beautifulsoup4. JS rendering requires Playwright (playwright install chromium).

Where does the fetched data live?

The server fetches data directly and returns it to the AI assistant in the response. No data is stored or persisted by the server itself. Response length can be capped with max_bytes.

What transports or authentication does webfetch-mcp support?

It communicates with MCP-compatible assistants over stdio (the command-line invocation). Authentication is handled by domain-scoped headers set in the YAML config file.

Are there any known limits of webfetch-mcp?

The README does not document explicit functional limits. It supports response truncation via max_bytes to avoid filling the assistant’s context window.

「その他」の他のコンテンツ

🚀 Model Context Protocol (MCP) Curriculum for Beginners

microsoft

This open-source curriculum introduces the fundamentals of Model Context Protocol (MCP) through real-world, cross-language examples in .NET, Java, TypeScript, JavaScript, Rust and Python. Designed for developers, it focuses on practical techniques for building modular, scalable,

16.7K