MCP.so
Sign In

MCP Web Search Crawler

@tomas-hanzlik

Repository

About MCP Web Search Crawler

A lightweight MCP server for web searching via DuckDuckGo and extracting page content as Markdown.

#mcp#web-crawler#duckduckgo

Basic information

Category

Search

Runtime

python

Transports

stdio

Publisher

tomas-hanzlik

Config

Add this server to your MCP-compatible client using the configuration below.

{
  "mcpServers": {
    "mcp-web-search-crawl": {
      "command": "uvx",
      "args": [
        "--from",
        "git+https://github.com/tomas-hanzlik/mcp-web-search-crawl",
        "mcpsearchcrawl"
      ]
    }
  }
}

Tools

No tools detected

We auto-extract tools from the README. The maintainer can list them under a ## Tools heading to populate this section.

Overview

What is MCP Web Search Crawler?

MCP Web Search Crawler is a Model Context Protocol server that provides web search and crawling capabilities. It enables AI assistants and other MCP clients to search the web using DuckDuckGo and crawl specific URLs to extract content in markdown format.

How to use MCP Web Search Crawler?

Clone the repository, then configure the server in your MCP client using the uv command with a directory argument, or use uvx from the GitHub repository. It exposes a tool called search_links that accepts a query parameter. The server can be run with --transport sse or --transport stdio.

Key features of MCP Web Search Crawler

  • Web Search via DuckDuckGo returning links, titles, and body snippets
  • URL crawling returning content as markdown
  • Configurable via MCP client JSON
  • Programmatic testing with Python client (fastmcp)
  • Supports SSE and stdio transports

Use cases of MCP Web Search Crawler

  • AI assistants searching the web for current information
  • Crawling specific web pages to extract markdown content
  • Combining search and crawl for research tasks
  • Integrating with MCP-compatible clients for automated web data retrieval

FAQ from MCP Web Search Crawler

What dependencies are required?

Requires uv and Python. The server uses DuckDuckGo as its search backend and does not need API keys.

How do I test the server programmatically?

Use the provided Python script with the fastmcp Client, then call tools like search_links with a query parameter.

What transport options are available?

The server supports SSE and stdio transports. Run it with --transport sse or --transport stdio.

How can I debug the server?

Use MCP Inspector with the command: uv run fastmcp dev src/mcp_web_search_crawl/server.py:mcp --with-editable .

Comments

More Search MCP servers

🚀 OneSearch MCP Server: Web Search & Crawl & Scraper & Extract

yokingma

🚀 OneSearch MCP Server: Web Search & Scraper & Extract, Support agent-browser, SearXNG, Tavily, DuckDuckGo, Bing, etc.

127

SerpApi MCP Server

ilyazub

SerpApi MCP Server for Google and other search engine results

154

Google Search Tool

web-agent-master

A Playwright-based Node.js tool that bypasses search engine anti-scraping mechanisms to execute Google searches. Local alternative to SERP APIs with MCP server integration.

611

Google News MCP Server

ChanMeng666

【Star-crossed coders unite!⭐️】Model Context Protocol (MCP) server implementation providing Google News search capabilities via SerpAPI, with automatic news categorization and multi-language support.

124

Version 2.0 is here!!

mixelpixx

MCP Server built for use with Claude Code, Claude Desktop, VS Code, Cline - enable google search and ability to follow links and research websites

249