Better Fetch

What is Better Fetch?

Better Fetch is a Model Context Protocol (MCP) server that intelligently fetches and processes web content with nested URL crawling capabilities. It transforms documentation sites and web resources into clean, structured markdown files suitable for AI consumption, content analysis, and knowledge base creation.

How to use Better Fetch?

Clone the repository, run npm install and npm run build, then configure your MCP client (e.g., Claude Desktop or VS Code MCP extension) with the path to dist/server.js. Use the tools fetch_website_nested for multi-page crawling or fetch_website_single for single-page extraction.

Key features of Better Fetch

• Smart web crawling with nested URL fetching and configurable depth
• Single page mode for simple content extraction
• Domain filtering and regex pattern matching
• Intelligent content cleaning that removes ads, navigation, and scripts
• Automatic generation of table of contents with proper nesting
• Clean markdown output with code block and link preservation
• Highly configurable crawl depth, page limits, and timeouts

Use cases of Better Fetch

• Convert API documentation, framework guides, and tutorial series into organized markdown
• Gather competitor documentation and feature descriptions for competitive analysis
• Prepare clean, structured content for LLM training or RAG system input
• Build comprehensive knowledge bases for customer service chatbots
• Automate web-based research material collection and summarization

FAQ from Better Fetch

What are the prerequisites for using Better Fetch?

Node.js 18+, npm or yarn, and an MCP-compatible client such as Claude Desktop or VS Code with an MCP extension.

What tools does Better Fetch provide?

It provides two tools: fetch_website_nested for multi‑page crawling with options like max depth, max pages, domain filtering, and pattern matching; and fetch_website_single for extracting a single page.

What are the default crawl limits?

Default max depth is 2, max pages is 50, request timeout is 10 seconds, and there is a 500ms delay between requests.

Is Better Fetch open source?

Yes, it is licensed under the MIT License.

How do I configure Better Fetch for Claude Desktop?

Add an entry to claude_desktop_config.json with command "node", args pointing to the absolute path of dist/server.js, and env {"NODE_ENV": "production"}.