What it is
A read-only MCP server for a live benchmark of four web search APIs for AI agents (You.com, Tavily, Exa and Parallel) on 30 agent-style queries across GTM, insurance and finance. Engineers choosing a search API for an agent can ask for scorecards, side-by-side comparisons, per-query answer keys, charts and methodology, then rerun the benchmark themselves. Free, no sign-in. Nothing calls a search API or stores data; results are the published snapshot from the Oct 8, 2026 run.
From the publisher's website: You.com, Tavily, Exa and Parallel compared live on 30 agent-style queries across GTM, insurance and finance: accuracy, freshness, latency and setup friction.
Connect
- Endpoint:
https://websearch-api-eval.psberkeleymba.workers.dev/mcp (Streamable HTTP)
- Authentication: no authentication
Claude Code:
claude mcp add web-search-api-eval --transport http https://websearch-api-eval.psberkeleymba.workers.dev/mcp
Other clients (mcpServers config):
{ "mcpServers": { "web-search-api-eval": { "type": "http", "url": "https://websearch-api-eval.psberkeleymba.workers.dev/mcp" } } }
Tools
Read from the server's tools/list on 2026-11-13:
get_overall_results — Headline findings and the overall scorecard across all 30 queries (90 calls per API): accuracy, fully correct, precision@5, freshness, empty responses, failed calls, latency p50/p95, cost per correct answer
get_scorecard — Full metric scorecard for one use case (gtm, insurance, finance) or 'overall'
compare_providers — Compare two or more of You.com, Tavily, Exa and Parallel side by side, overall or within one use case, optionally filtered to metrics whose name contains a keyword (e.g
recommend_provider — Which web search API fits an agent, based on this benchmark: live voice/chat turns, deep research, semantic or long-tail queries, docs lookups, or a specific use case (GTM, insurance, finance)
list_questions — The 30 benchmark questions (G1-G10 GTM, I1-I10 insurance, F1-F10 finance): ID, what the agent is asked, the search call and freshness
get_question — One benchmark question by ID (e.g
get_charts — The published graphs: overall scorecard, key metrics and per-query grids for GTM, insurance and finance, and developer activation
get_methodology — How the benchmark was run and graded: query design, identical request parameters, 3 runs per query, latency timing, SimpleQA and FreshQA grading, the 10 metrics and the standards they follow, and limits
how_to_run_locally — How to rerun this benchmark on your own machine with your own API keys, including on your own queries
search — Search the Web Search API Eval benchmark (results, questions, methodology, charts, how to run)
fetch — Full text of one benchmark document by its ID (for example question-F5, scorecard-finance, overall, methodology)
Notes
- Description above is the publisher's own summary from their submission; check the website for plans, pricing and limits.
- The endpoint answered a standard MCP
initialize request when checked on 2026-11-13.
Sources: Website, Submission.
Comments