广告合作
MCP.so
登录

MCP Server Webcrawl

@MCP-Mirror

代码仓库

基本信息

分类

其他

许可证

View license

传输方式

stdio

作者

MCP-Mirror

关于 MCP Server Webcrawl

Mirror of

#mcp-server#web-crawl#ai-integration

MCP Server Webcrawl 配置

暂无标准配置

该服务器的 README 中没有可解析的 MCP 配置块，请前往代码仓库查看安装说明。

代码仓库

MCP Server Webcrawl 工具

工具列表暂未提供

当前收录暂时没有可展示的工具列表，不代表服务没有工具。请查阅项目文档了解支持的能力。

工具是从 README 中自动提取的。维护者可以在 ## Tools 标题下列出工具,即可填充这部分内容。

概览

What is mcp-server-webcrawl?

mcp-server-webcrawl bridges the gap between your web crawl and AI language models using the Model Context Protocol (MCP). It allows an AI client to filter and analyze web content under your direction or autonomously, and works with a variety of web crawlers (WARC, wget, InterroBot, Katana, SiteOne).

How to use mcp-server-webcrawl?

Install via pip install mcp-server-webcrawl (requires Python ≥3.10 and Claude Desktop). Configure the MCP server in Claude Desktop’s settings file by adding an entry under mcpServers with the command mcp-server-webcrawl and arguments specifying the crawler type and data source path. On macOS, use the absolute path to the executable.

Key features of mcp-server-webcrawl

  • Claude Desktop ready
  • Full-text search with boolean support
  • Filter resources by type, HTTP status, and more
  • Compatible with multiple web crawlers
  • Quick MCP configuration
  • ChatGPT support coming soon

Use cases of mcp-server-webcrawl

  • Search and analyze crawled web archives with natural language queries
  • Filter web content by file type or HTTP status for targeted analysis
  • Enable AI assistants to browse and summarize local crawl data
  • Integrate with various crawler outputs (WARC, wget, etc.) in one tool

常见问题

1

What crawlers are supported by mcp-server-webcrawl?

mcp-server-webcrawl works with WARC, wget, InterroBot, Katana, and SiteOne.

2

How do I install mcp-server-webcrawl?

Install via `pip install mcp-server-webcrawl`. It requires Python 3.10 or higher and Claude Desktop.

3

How do I configure mcp-server-webcrawl for macOS?

On macOS, you must use the absolute path to the `mcp-server-webcrawl` executable in the `command` field (e.g., `/Users/yourusername/.local/bin/mcp-server-webcrawl`). Run `which mcp-server-webcrawl` to find the correct path.

4

What is the data source argument for mcp-server-webcrawl?

The `--datasrc` argument specifies the directory or database path for the crawled content. Its value depends on the crawler: for wget it’s the parent directory of mirrors; for WARC it’s the parent of WARC files; for InterroBot it’s the direct database path; for Katana and SiteOne it’s the parent directory of archives.

5

What transport does mcp-server-webcrawl use?

mcp-server-webcrawl uses stdio transport as configured in Claude Desktop’s MCP settings via the `command` and `args` fields.

评论

其他 分类下的更多 MCP 服务器

Reactive Resume

amruthpillai

A one-of-a-kind resume builder that keeps your privacy in mind. Completely secure, customizable, portable, open-source and free forever. Try it out today!

39.5K

Sequential Thinking

modelcontextprotocol

Model Context Protocol Servers

88.2K

DeepMark

DeepMark helps teachers deliver rapid, consistent marking with meaningful feedback for every student — in a fraction of the time. What once took a week, now takes one free period.

Klarix Intelligence Engine

Klarix

B2B competitive intelligence, live sales battlecards, 5-axis ICP prospect fit scoring, and tech stack teardowns over MCP and REST with cited public-web evidence.

Maestro

mobile-dev-inc

Painless E2E Automation for Mobile and Web

14.8K
广告合作