11 #crawler MCP Servers & Clients

Web Scout MCP Server

pinkpixel-dev

A powerful MCP server extension providing web search and content extraction capabilities. Integrates DuckDuckGo search functionality and URL content extraction into your MCP environment, enabling AI assistants to search the web and extract webpage content programmatically.

Documentation Crawler & MCP Server

alizdavoodi

This project provides a toolset to crawl websites wikis, tool/library documentions and generate Markdown documentation, and make that documentation searchable via a Model Context Protocol (MCP) server, designed for integration with tools like Cursor.

AIO-MCP Server

athapong

🚀 All-in-one MCP server with AI search, RAG, and multi-service integrations (GitLab/Jira/Confluence/YouTube) for AI-enhanced development workflows. Folk from https://github.com/nguyenvanduocit/all-in-one-model-context-protocol

mcp-server-requests

coucya

Web Content Retrieval (full webpage, filtered content, or Markdown-converted), Custom User-Agent, Multi-HTTP Method Support (GET/POST/PUT/DELETE/PATCH), LLM-Controlled Request Headers, LLM-Accessible Response Headers, and more.

LeetCode Crawler

louisfghbvc

mcp server for craw the interview questions in discussion

Infomesh

dotnetpower

🕸️ Fully decentralized P2P search engine for LLMs — no API key, no billing, forever free. MCP-native web search via Kademlia DHT + libp2p + FTS5. Production stable ✅ → pip install infomesh

Better Fetch

flutterninja9

A powerful Model Context Protocol (MCP) server that intelligently fetches and processes web content with nested URL crawling capabilities. Transform any documentation site or web resource into clean, structured markdown files perfect for AI consumption and analysis.

Web Scout MCP 🌐

suebksnn

A powerful MCP server extension providing web search and content extraction capabilities. Integrates DuckDuckGo search functionality and URL content extraction into your MCP environment, enabling AI assistants to search the web and extract webpage content programmatically.

Webpeel

webpeel

Web data API that just works. Pass any URL, get clean markdown — Cloudflare, JavaScript, CAPTCHAs handled automatically. No config needed. 97.6% success rate across 500+ domains. Free tier included.

crawl4ai-mcp

wyattowalsh

MCP (Model Context Protocol) server wrapping Crawl4AI library as functions using Python