AI-First Scraper MCP Server

@yubinkim444

リポジトリ

基本情報

カテゴリ

ブラウザ自動化

トランスポート

stdio

作者

yubinkim444

投稿者

YooBin Kim

AI-First Scraper MCP Server について

Ad-free web scraping and search exposed as 3 MCP tools

#web-scraping #markdown #search #ai-agents #rag

AI-First Scraper MCP Server の設定

以下の設定を使って、このサーバーを MCP 対応クライアントに追加してください。

{
  "mcpServers": {
    "ai-first-scraper": {
      "command": "uvx",
      "args": [
        "ai-first-scraper-mcp"
      ]
    }
  }
}

AI-First Scraper MCP Server のツール

ツール一覧はまだありません

この掲載には表示できるツール一覧がまだありません。サーバーにツールがないという意味ではありません。対応機能はプロジェクトのドキュメントをご確認ください。

上のボタンで一時的なサンドボックスでこのサーバーを実行し、ツール一覧を取得できます。

概要

What is AI-First Scraper?

AI-First Scraper is an MCP server that adds three web-scraping and search tools to any MCP-compatible agent (Claude Desktop, Cursor, Cline). It fetches URLs and returns clean Markdown, stripping ads and HTML clutter, and can also run web searches that return top results already converted to Markdown.

How to use AI-First Scraper?

Install via uvx (no install, runs from PyPI) or pip install ai-first-scraper-mcp. Add the server configuration to your client’s settings JSON (e.g., claude_desktop_config.json, ~/.cursor/mcp.json, or cline_mcp_settings.json). Restart the client; the three tools (fetch_page, fetch_pages_batch, search_web) appear automatically.

Key features of AI-First Scraper

fetch_page: convert one URL to clean Markdown (HTML or PDF).
fetch_pages_batch: fetch up to 25 URLs in parallel, return Markdown.
search_web: run a web search and return top-k results as Markdown.
Backed by ai-first-scraper and ai-first-search APIs.
Supports self‑hosted backends via environment variables.
Requires Python 3.10+.

Use cases of AI-First Scraper

Let an AI agent summarize recent articles without raw HTML.
Batch‑scrape multiple pages for competitive analysis.
Search the web and immediately pass clean content to an LLM.
Replace manual curl+parsing workflows in MCP‑powered automation.

よくある質問

What tools does AI-First Scraper provide?

It provides `fetch_page` (single URL to Markdown), `fetch_pages_batch` (up to 25 URLs in parallel to Markdown), and `search_web` (web search with Markdown results).

How do I install AI-First Scraper?

The fastest method is `uvx` (no install). Alternatively, run `pip install ai-first-scraper-mcp` and then set the command to `ai-first-scraper-mcp` in your MCP client config.

Can I run my own backend instead of the public APIs?

Yes. Set environment variables `SCRAPER_URL` and `SEARCH_URL` in the MCP server config to point at your self‑hosted instances. You can also set `AFS_TIMEOUT`.

Where do I put the configuration file?

For Claude Desktop (macOS): `~/Library/Application Support/Claude/claude_desktop_config.json`. For Claude Desktop (Windows): `%APPDATA%\Claude\claude_desktop_config.json`. For Cursor: `~/.cursor/mcp.json`. For Cline (VS Code): `~/Library/Application Support/Code/User/globalStorage/saoudrizwan.claude-dev/settings/cline_mcp_settings.json`.

What dependencies does AI-First Scraper have?

It requires Python 3.10 or higher and uses the external ai-first-scraper and ai-first-search APIs (public or self‑hosted). The server communicates over MCP via stdio.

「ブラウザ自動化」の他のコンテンツ

fetch-mcp

egoist

An MCP server for fetching URLs / Youtube video transcript.

156

Fetch

modelcontextprotocol

Model Context Protocol Servers

88.1K

Fastcrw

Fast, lightweight Firecrawl/Tavily alternative in Rust. Web scraper, crawler & search API with MCP server for AI agents. Drop-in Firecrawl-compatible API (/scrape, /crawl, /search). 2.3x faster than Tavily, 1.5x faster than Firecrawl in 1K-URL benchmarks. 6 MB RAM, single binary.

286