Crawler For Llm

Name: Crawler For Llm
Author: KonghaYao

@KonghaYao

About Crawler For Llm

高等级的免费爬虫MCP, 完全 Markdown 化支持，支持微信公众号等一大批国内网站。A powerful web crawler designed specifically for LLM applications, capable of extracting clean, readable content from various web pages and converting it to Markdown format. This tool is essential for building knowledge bases, training

#web-crawler #markdown-converter #content-extraction #llm-data-preparation #knowledge-base-building #model-context-protocol-mcp

Basic information

Config

Add this server to your MCP-compatible client using the configuration below.

{
  "mcpServers": {
    "langgraph-crawler": {
      "command": "npx",
      "args": [
        "-y",
        "@langgraph-js/crawler-mcp@latest"
      ]
    }
  }
}

Tools

A powerful web content extraction tool that retrieves and processes raw content from specified URLs, ideal for data collection, content analysis, and research tasks.

A powerful web search tool that provides comprehensive, real-time results using search engine. Returns relevant web content with customizable parameters for result count, content type, and domain filtering. Ideal for gathering current information, news, and detailed web content analysis.

Overview

What is Crawler For Llm?

Crawler For Llm is a universal web crawler designed for LLM applications, extracting clean, readable content from web pages and converting it to Markdown. It includes built-in support for the Model Context Protocol (MCP) for enhanced context management, and is intended for building knowledge bases, training data collection, and content aggregation.

How to use Crawler For Llm?

Add the provided MCP server configuration to your project's settings. Use the command npx -y @langgraph-js/crawler-mcp@latest with the server name langgraph-crawler.

Key features of Crawler For Llm

Universal web crawler with MCP integration
Extracts clean content and converts to Markdown
Supports documentation, development, and Chinese platforms
Continuously expanding list of supported websites
Built for LLM knowledge bases and training data

Use cases of Crawler For Llm

Building knowledge bases for LLM applications
Collecting training data from diverse web sources
Aggregating content from multiple platforms

FAQ from Crawler For Llm

—

Comments

More Browser Automation MCP servers

mcp-browser-use: MCP server for browser-use

vinayak-mehta

MCP server for browser-use

BrowserTools MCP

AgentDeskAI

Monitor browser logs directly from Cursor and other MCP compatible IDEs.

7.3K

Slack Slack

microsoft

Playwright is a framework for Web Testing and Automation. It allows testing Chromium, Firefox and WebKit with a single API.

92.5K

Puppeteer MCP Server

merajmehrabi

This MCP server provides browser automation capabilities through Puppeteer, allowing interaction with both new browser instances and existing Chrome windows.

474

Playwright Mcp