13 件の#crawler MCP サーバー & クライアント

Web Scout MCP Server

pinkpixel-dev

A powerful MCP server extension providing web search and content extraction capabilities. Integrates DuckDuckGo search functionality and URL content extraction into your MCP environment, enabling AI assistants to search the web and extract webpage content programmatically.

Documentation Crawler & MCP Server

alizdavoodi

This project provides a toolset to crawl websites wikis, tool/library documentions and generate Markdown documentation, and make that documentation searchable via a Model Context Protocol (MCP) server, designed for integration with tools like Cursor.

AIO-MCP Server

athapong

🚀 All-in-one MCP server with AI search, RAG, and multi-service integrations (GitLab/Jira/Confluence/YouTube) for AI-enhanced development workflows. Folk from https://github.com/nguyenvanduocit/all-in-one-model-context-protocol

mcp-server-requests

coucya

Web Content Retrieval (full webpage, filtered content, or Markdown-converted), Custom User-Agent, Multi-HTTP Method Support (GET/POST/PUT/DELETE/PATCH), LLM-Controlled Request Headers, LLM-Accessible Response Headers, and more.

LeetCode Crawler

louisfghbvc

mcp server for craw the interview questions in discussion

Infomesh

dotnetpower

🕸️ Fully decentralized P2P search engine for LLMs — no API key, no billing, forever free. MCP-native web search via Kademlia DHT + libp2p + FTS5. Production stable ✅ → pip install infomesh

Web Scout MCP 🌐

suebksnn

A powerful MCP server extension providing web search and content extraction capabilities. Integrates DuckDuckGo search functionality and URL content extraction into your MCP environment, enabling AI assistants to search the web and extract webpage content programmatically.

Better Fetch

flutterninja9

A powerful Model Context Protocol (MCP) server that intelligently fetches and processes web content with nested URL crawling capabilities. Transform any documentation site or web resource into clean, structured markdown files perfect for AI consumption and analysis.

Crawl Readiness

Grubbomatic

AI SEO tools for MCP clients — audit whether AI crawlers can read a website, validate JSON-LD and robots.txt, and generate llms.txt, AI-crawler-aware robots.txt, and JSON-LD schema without leaving the conversation.

crawl4ai-mcp

wyattowalsh

MCP (Model Context Protocol) server wrapping Crawl4AI library as functions using Python

Webpeel

webpeel

Web data API that just works. Pass any URL, get clean markdown — Cloudflare, JavaScript, CAPTCHAs handled automatically. No config needed. 97.6% success rate across 500+ domains. Free tier included.

QuanticData — Web Scraping MCP Server

QuanticData

Web scraping MCP server for AI agents: scrape any page to clean Markdown, SERP search (Google, Bing, DuckDuckGo), site crawl and map, 74 ready-made collectors and residential proxies. 25 tools, pay per success.