Web Speed

Name: Web Speed
Author: Dominic-Pi-Sunyer

@Dominic-Pi-Sunyer

About Web Speed

One call turns any web page into a structured map — what it is, what's on it, and the actions an agent can take (search, add-to-cart, log in, paginate) — instead of raw HTML or screenshots. Served from a shared, always-current registry, so popular pages return in milliseconds.

#web #agent #scraping #browser #automation #search #data #api #llm

Basic information

Config

Add this server to your MCP-compatible client using the configuration below.

{
  "mcpServers": {
    "webspeed": {
      "command": "npx",
      "args": [
        "-y",
        "mcp-remote@latest",
        "https://api.getwebspeed.io/mcp/sse",
        "--header",
        "X-Web-Speed-Key: wsp_your_key_here"
      ]
    }
  }
}

Tools

Full structured map: headings, navigation, content links, forms, tables, text, metadata

Submit a form (GET or POST), get back the resulting page's map

Crawl from a root URL, return a combined map of all pages

Deep structural data for nodes matching a CSS selector

Instant page classification — `login`, `listing`, `article`, `form`, `navigation`, `other`

Drop a cached map so the next call fetches fresh

Overview

What is Web Speed?

Web Speed solves the Signal‑to‑Noise problem for AI agents by translating cluttered, human‑oriented web HTML into a deterministic, token‑efficient structural map. It contains no AI itself—all interpretation is left to the calling agent—and provides six tools (interpret_page, submit_form, site_map, inspect_element, page_type, invalidate_cache) via the Model Context Protocol.

How to use Web Speed?

Install by cloning the repository, creating a Python virtual environment, and installing dependencies with pip install -r requirements.txt. Run locally with mcp dev server.py or over stdio with python server.py. Register with Claude Desktop or Cowork by adding a server entry to the MCP client configuration JSON; the six tools then appear under the web-speed server.

Key features of Web Speed

Strips non‑structural content for up to 97% token reduction.
Returns a frozen structural map—no hallucinated element IDs.
Same JSON schema for every website, eliminating per‑site scrapers.
site_map crawls an entire domain in a single call.
Built‑in 24‑hour disk cache; optional crowdsourced registry sync.

Use cases of Web Speed

Navigating a site by reading navigation and content links, then fetching specific pages.
Submitting forms (GET or POST) with CSRF tokens preserved verbatim.
Classifying a page instantly (login, listing, article, etc.) before deeper analysis.
Drilling into specific elements via CSS selectors with inspect_element.
Pre‑planning multi‑step workflows using the full domain map from site_map.

FAQ from Web Speed

What runtime or dependencies does Web Speed require?

A Python 3 environment with the packages listed in requirements.txt. No LLM dependencies are needed—all AI logic lives in the calling agent.

Where does fetched page data live?

Maps are cached as JSON files in a local ./cache/ directory with a 24‑hour TTL. Optionally, structural page data (not form values or session tokens) is contributed to the Web Speed shared registry at api.getwebspeed.io. This sync is opt‑out via the WEB_SPEED_REGISTRY_SYNC=false environment variable.

What are the known limitations of Web Speed?

JS‑rendered SPAs (React, Vue, Angular) return only the pre‑render HTML shell—content injected by JavaScript will be missing. Page type classification is heuristic and may be inaccurate for shell‑only SPAs. Elements returned by inspect_element are capped at 25; navigation and content links are each capped at 60.

How are tools invoked if an error occurs?

Tools never raise exceptions. On failure they return a JSON object with "error": true, a code (FETCH_FAILED, PARSE_FAILED, TIMEOUT, or NOT_HTML), and a message with a human‑readable explanation.

Is authentication or authorization handled by Web Speed?

The README does not describe any built‑in authentication or authorization mechanisms. Form fields, including hidden CSRF tokens, are passed back verbatim for the calling agent to handle.