Where is the source code for TheCrawler?
The source code is linked from the repository URL on this page.
@manchittlab
基本信息
Open-source web scraper + LLM-powered structured extraction. PDF/DOCX, markdown, JSON-LD, microdata, commerce data, forms, 16 analytics-tracker detection. Structured errors with retryable flags. Adaptive Cheerio->Playwright. CLI, npm, REST API, and MCP server. AGPL-3.0.
使用下面的配置,将此服务器添加到你的 MCP 客户端。
{
"mcpServers": {
"TheCrawler": {
"command": "node",
"args": [
"dist/cli.js",
"crawl",
"https://example.com",
"--markdown"
]
}
}
}工具列表暂未提供
当前收录暂时没有可展示的工具列表,不代表服务没有工具。请查阅项目文档了解支持的能力。
工具是从 README 中自动提取的。维护者可以在 ## Tools 标题下列出工具,即可填充这部分内容。
Open-source web scraper + LLM-powered structured extraction. PDF/DOCX, markdown, JSON-LD, microdata, commerce data, forms, 16 analytics-tracker detection. Structured errors with retryable flags. Adaptive Cheerio->Playwright. CLI, npm, REST API, and MCP server. AGPL-3.0.
The README includes setup instructions such as node dist/cli.js crawl https://example.com --markdown.
The source code is linked from the repository URL on this page.
If the README contains a parseable MCP configuration block, it is shown in the Config tab.

Parse allows your agent to access thousands of APIs instantly. You can use Parse to build an API for any action on any website and edit any previously existing API in our catalogue.

Model Context Protocol Servers
Capture screenshots of webpages as images or PDFs with Screenshot Scout.
Loadster is a load testing, stress testing, and synthetic monitoring platform. Run Playwright, Browser Bots, and Protocol Bots to test and monitor your websites, web apps, and APIs. Loadster MCP gives your agents the abi

MCP server for managing Gologin browser profiles, proxies, fingerprints and cloud browser sessions through AI conversations
评论