Advertise
MCP.so
Sign In

MCP Server Webcrawl

@pragmar

RepositoryHomepage

Basic information

Category

Other

License

NOASSERTION

Transports

stdio

Author

pragmar

About MCP Server Webcrawl

MCP server tailored to connecting web crawler data and archives

#mcp#wget#knowledgebase#warc#llm#mcp-servers#mcp-server

MCP Server Webcrawl config

No standard config provided

This server doesn't expose a parseable MCP config block in its README. See the repository for install instructions.

Repository

MCP Server Webcrawl tools

Tool list not available yet

This listing does not currently have a tool list to display. This does not mean the server has no tools; check its documentation for supported capabilities.

We auto-extract tools from the README. The maintainer can list them under a ## Tools heading to populate this section.

Overview

What is mCP Server WebCrawl?

mcp-server-webcrawl provides advanced search and retrieval for web crawler data. It allows AI clients to filter and analyze web content via a fulltext search with boolean support and resource filtering by type, HTTP status, and more. It is designed for users of Claude Desktop who work with data from crawlers like ArchiveBox, HTTrack, or wget.

How to use mCP Server WebCrawl?

mcp-server-webcrawl requires Python (>=3.10) and is installed via pip install mcp-server-webcrawl. After installation, it is configured as an MCP server for Claude Desktop. It can also run in interactive mode as a terminal search tool for web archives without an AI client.

Key features of mCP Server WebCrawl

  • Claude Desktop ready
  • Multi-crawler compatible (ArchiveBox, wget, and more)
  • Filter resources by type, HTTP status, and more
  • Boolean search with field-specific and fulltext queries
  • Support for Markdown, snippets, regex, and XPath extras

Use cases of mCP Server WebCrawl

  • SEO audits of crawled website content
  • Broken link detection and pattern analysis
  • Website performance and optimization analysis
  • File organization and asset composition review
  • Searchable web archive in a terminal via interactive mode

Frequently asked questions

1

What web crawlers does mCP Server WebCrawl work with?

It works with ArchiveBox, HTTrack, InterroBot, Katana, SiteOne, WARC format, and wget.

2

What are the system requirements for mCP Server WebCrawl?

It requires Python version 3.10 or higher and the Claude Desktop application.

3

Is mCP Server WebCrawl free and open source?

Yes, it is free and open source.

4

What search syntax does mCP Server WebCrawl support?

It supports boolean operators (AND, OR, NOT), exact phrase matching, wildcards, field-specific searches (e.g., `type: html`, `status: 200`), and complex grouped expressions.

Comments

More Other MCP servers

Soar Flight Booking MCP

Soar Flight Booking MCP is a remote MCP server available at https://mcp.soar.flights.

Senso MCP for Joomla

sensomedia

Senso MCP turns a Joomla 5 or 6 site into a remote MCP server. It runs inside Joomla itself (no extra server, no Node.js bridge) and is secured with OAuth 2.1 and Joomla's own ACL. The free pack lets an AI assistant crea

DeepMark

DeepMark helps teachers deliver rapid, consistent marking with meaningful feedback for every student — in a fraction of the time. What once took a week, now takes one free period.

YOUB

Connect your AI assistant to your own YOUB endurance training account. Read recorded activities with full analysis, planned sessions, race goals, wellness (HRV, sleep, resting HR) and your athletic profile. Create or mov

Lemon.io

lemon-io

Request dedicated senior developers to work on your project, write job descriptions, and prep technical interviews — all without leaving your Claude chat. Just describe what you need, and Lemon.io MCP does the rest. 1–3

Advertise