Crawler For Llm MCP Server

Name: Crawler For Llm MCP Server
Author: KonghaYao

@KonghaYao

リポジトリ

基本情報

カテゴリ

ブラウザ自動化

トランスポート

stdio

作者

KonghaYao

投稿者

心屿江

Crawler For Llm MCP Server について

高等级的免费爬虫MCP, 完全 Markdown 化支持，支持微信公众号等一大批国内网站。A powerful web crawler designed specifically for LLM applications, capable of extracting clean, readable content from various web pages and converting it to Markdown format. This tool is essential for building knowledge bases, training

#web-crawler #markdown-converter #content-extraction#llm-data-preparation#knowledge-base-building#model-context-protocol-mcp

Crawler For Llm MCP Server の設定

以下の設定を使って、このサーバーを MCP 対応クライアントに追加してください。

{
  "mcpServers": {
    "langgraph-crawler": {
      "command": "npx",
      "args": [
        "-y",
        "@langgraph-js/crawler-mcp@latest"
      ]
    }
  }
}

Crawler For Llm MCP Server のツール

A powerful web content extraction tool that retrieves and processes raw content from specified URLs, ideal for data collection, content analysis, and research tasks.

A powerful web search tool that provides comprehensive, real-time results using search engine. Returns relevant web content with customizable parameters for result count, content type, and domain filtering. Ideal for gathering current information, news, and detailed web content analysis.

概要

What is Crawler For Llm?

Crawler For Llm is a universal web crawler designed for LLM applications, extracting clean, readable content from web pages and converting it to Markdown. It includes built-in support for the Model Context Protocol (MCP) for enhanced context management, and is intended for building knowledge bases, training data collection, and content aggregation.

How to use Crawler For Llm?

Add the provided MCP server configuration to your project's settings. Use the command npx -y @langgraph-js/crawler-mcp@latest with the server name langgraph-crawler.

Key features of Crawler For Llm

Universal web crawler with MCP integration
Extracts clean content and converts to Markdown
Supports documentation, development, and Chinese platforms
Continuously expanding list of supported websites
Built for LLM knowledge bases and training data

Use cases of Crawler For Llm

Building knowledge bases for LLM applications
Collecting training data from diverse web sources
Aggregating content from multiple platforms

FAQ from Crawler For Llm

—

「ブラウザ自動化」の他のコンテンツ

Playwright Mcp

microsoft

Playwright MCP server

34.8K

Chrome DevTools for agents

ChromeDevTools

Official Chrome DevTools MCP server from Google that lets coding agents control, debug and profile a live Chrome browser.

53.6K

BrowserTools MCP

AgentDeskAI

Monitor browser logs directly from Cursor and other MCP compatible IDEs.

7.3K

Scrapling Fetch MCP

cyberchitta

Helps AI assistants access text content from bot-protected websites. MCP server that fetches HTML/markdown from sites with anti-automation measures using Scrapling.

101

Fastcrw

Fast, lightweight Firecrawl/Tavily alternative in Rust. Web scraper, crawler & search API with MCP server for AI agents. Drop-in Firecrawl-compatible API (/scrape, /crawl, /search). 2.3x faster than Tavily, 1.5x faster than Firecrawl in 1K-URL benchmarks. 6 MB RAM, single binary.

286