MCP Website Downloader

What is MCP Website Downloader?

MCP Website Downloader is an MCP server for downloading documentation websites and preparing them for RAG indexing. It is intended for users who want to capture website content for offline use or retrieval-augmented generation (RAG) systems.

How to use MCP Website Downloader?

Install by cloning the repository, creating a virtual environment with uv venv, activating it, and running pip install -e .. Configure the server in your MCP client’s config file (e.g., claude_desktop_config.json) with the command uv run mcp-windows-website-downloader --library <path>. Alternatively, start the server directly with python -m mcp_windows_website_downloader.server --library docs_library and use MCP client tools to call the download tool with a URL.

Key features of MCP Website Downloader

• Downloads documentation websites for offline use
• Maintains original site structure and navigation
• Downloads and organizes CSS, JS, and image assets
• Creates a rag_index.json index for RAG systems
• Handles common errors like invalid URLs and network failures

Use cases of MCP Website Downloader

• Preparing documentation sites for RAG indexing and retrieval
• Archiving web documentation for offline browsing
• Structuring website content for use with AI tools or language models

FAQ from MCP Website Downloader

What dependencies does MCP Website Downloader require?

Python and uv are required. The server uses pip to install the package and expects an MCP client (e.g., Claude Desktop) to invoke its tools.

Where does MCP Website Downloader store downloaded content?

Content is stored in a local library folder specified by the --library argument. The folder is structured by domain, with HTML pages, assets (CSS, JS, images), and a rag_index.json index file.

How does MCP Website Downloader handle errors?

The server returns JSON error responses with status "error" and a detailed message. It handles invalid URLs, network errors, asset download failures, malformed HTML, deep recursion, and file system issues.

What transport does MCP Website Downloader use?

It uses the standard MCP interface, running as a subprocess communicated via stdio (as configured in the client’s JSON config).

Is there a limit to how many pages MCP Website Downloader can download?

The README mentions “reasonable depth limits” as a design principle and handles deep recursion in error handling, suggesting a configurable or built-in limit, but the exact value is not specified.