mcp-server-webcrawl

Name: mcp-server-webcrawl
Author: pragmar

@pragmar

リポジトリホームページ

mcp-server-webcrawl について

MCP server tailored to connecting web crawler data and archives

#mcp #wget #knowledgebase #warc #llm #mcp-servers #mcp-server

基本情報

カテゴリ

その他

ライセンス

NOASSERTION

ランタイム

python

トランスポート

stdio

公開者

pragmar

設定

標準の設定はありません

このサーバーの README には解析可能な MCP 設定ブロックが含まれていません。インストール手順はリポジトリをご確認ください。

リポジトリ

ツール

ツールは検出されませんでした

ツールは README から自動的に抽出されます。メンテナーは ## Tools という見出しの下に記載することで、このタブに反映できます。

概要

What is mCP Server WebCrawl?

mcp-server-webcrawl provides advanced search and retrieval for web crawler data. It allows AI clients to filter and analyze web content via a fulltext search with boolean support and resource filtering by type, HTTP status, and more. It is designed for users of Claude Desktop who work with data from crawlers like ArchiveBox, HTTrack, or wget.

How to use mCP Server WebCrawl?

mcp-server-webcrawl requires Python (>=3.10) and is installed via pip install mcp-server-webcrawl. After installation, it is configured as an MCP server for Claude Desktop. It can also run in interactive mode as a terminal search tool for web archives without an AI client.

Key features of mCP Server WebCrawl

Claude Desktop ready
Multi-crawler compatible (ArchiveBox, wget, and more)
Filter resources by type, HTTP status, and more
Boolean search with field-specific and fulltext queries
Support for Markdown, snippets, regex, and XPath extras

Use cases of mCP Server WebCrawl

SEO audits of crawled website content
Broken link detection and pattern analysis
Website performance and optimization analysis
File organization and asset composition review
Searchable web archive in a terminal via interactive mode

FAQ from mCP Server WebCrawl

What web crawlers does mCP Server WebCrawl work with?

It works with ArchiveBox, HTTrack, InterroBot, Katana, SiteOne, WARC format, and wget.

What are the system requirements for mCP Server WebCrawl?

It requires Python version 3.10 or higher and the Claude Desktop application.

Is mCP Server WebCrawl free and open source?

Yes, it is free and open source.

What search syntax does mCP Server WebCrawl support?

It supports boolean operators (AND, OR, NOT), exact phrase matching, wildcards, field-specific searches (e.g., type: html, status: 200), and complex grouped expressions.