Advertise
MCP.so
Sign In
A

AI-First Scraper MCP Server

@yubinkim444

Repository

Basic information

Category

Browser Automation

Transports

stdio

Author

yubinkim444

Submitted by

YooBin Kim

About AI-First Scraper MCP Server

Ad-free web scraping and search exposed as 3 MCP tools

#web-scraping#markdown#search#ai-agents#rag

AI-First Scraper MCP Server config

Add this server to your MCP-compatible client using the configuration below.

{
  "mcpServers": {
    "ai-first-scraper": {
      "command": "uvx",
      "args": [
        "ai-first-scraper-mcp"
      ]
    }
  }
}

AI-First Scraper MCP Server tools

Tool list not available yet

This listing does not currently have a tool list to display. This does not mean the server has no tools; check its documentation for supported capabilities.

Fetch the live tool list by running this server in a temporary sandbox using the button above.

Overview

What is AI-First Scraper?

AI-First Scraper is an MCP server that adds three web-scraping and search tools to any MCP-compatible agent (Claude Desktop, Cursor, Cline). It fetches URLs and returns clean Markdown, stripping ads and HTML clutter, and can also run web searches that return top results already converted to Markdown.

How to use AI-First Scraper?

Install via uvx (no install, runs from PyPI) or pip install ai-first-scraper-mcp. Add the server configuration to your client’s settings JSON (e.g., claude_desktop_config.json, ~/.cursor/mcp.json, or cline_mcp_settings.json). Restart the client; the three tools (fetch_page, fetch_pages_batch, search_web) appear automatically.

Key features of AI-First Scraper

  • fetch_page: convert one URL to clean Markdown (HTML or PDF).
  • fetch_pages_batch: fetch up to 25 URLs in parallel, return Markdown.
  • search_web: run a web search and return top-k results as Markdown.
  • Backed by ai-first-scraper and ai-first-search APIs.
  • Supports self‑hosted backends via environment variables.
  • Requires Python 3.10+.

Use cases of AI-First Scraper

  • Let an AI agent summarize recent articles without raw HTML.
  • Batch‑scrape multiple pages for competitive analysis.
  • Search the web and immediately pass clean content to an LLM.
  • Replace manual curl+parsing workflows in MCP‑powered automation.

Frequently asked questions

1

What tools does AI-First Scraper provide?

It provides `fetch_page` (single URL to Markdown), `fetch_pages_batch` (up to 25 URLs in parallel to Markdown), and `search_web` (web search with Markdown results).

2

How do I install AI-First Scraper?

The fastest method is `uvx` (no install). Alternatively, run `pip install ai-first-scraper-mcp` and then set the command to `ai-first-scraper-mcp` in your MCP client config.

3

Can I run my own backend instead of the public APIs?

Yes. Set environment variables `SCRAPER_URL` and `SEARCH_URL` in the MCP server config to point at your self‑hosted instances. You can also set `AFS_TIMEOUT`.

4

Where do I put the configuration file?

For Claude Desktop (macOS): `~/Library/Application Support/Claude/claude_desktop_config.json`. For Claude Desktop (Windows): `%APPDATA%\Claude\claude_desktop_config.json`. For Cursor: `~/.cursor/mcp.json`. For Cline (VS Code): `~/Library/Application Support/Code/User/globalStorage/saoudrizwan.claude-dev/settings/cline_mcp_settings.json`.

5

What dependencies does AI-First Scraper have?

It requires Python 3.10 or higher and uses the external ai-first-scraper and ai-first-search APIs (public or self‑hosted). The server communicates over MCP via stdio.

Comments

More Browser Automation MCP servers

QuantumProxies — Residential, Mobile & Datacenter Proxy MCP Server

quantumproxies

Give your AI agent its own proxies. QuantumProxies exposes residential, mobile, datacenter, ISP and IPv6 endpoints as MCP tools: list your plans, generate ready-to-use proxy strings with credentials, target by country, s

HTML/CSS to Image

An MCP server for generating images from HTML & CSS or screenshots of URLs using htmlcsstoimage.com.

Fetch MCP

jae-jae

MCP server for fetch web page content using Playwright headless browser.

1.1K

Fetcher MCP

jae-jae

MCP server for fetch web page content using Playwright headless browser.

1.1K

Nullprint

nullprintai

Let Claude Code, Claude Desktop, OpenAI Codex, Kimi and any MCP-capable AI agent create, launch, check and drive browser profiles in Nullprint, the AI antidetect browser. Install with one command.

Advertise