Web Content Extractor (agent Optimized)

Name: Web Content Extractor (agent Optimized)
Author: agenson-tools

@agenson-tools

Repository Homepage

About Web Content Extractor (agent Optimized)

Agent-optimized MCP server for extracting clean, structured content from web pages

Basic information

Config

Add this server to your MCP-compatible client using the configuration below.

{
  "mcpServers": {
    "web-content-extractor": {
      "command": "npx",
      "args": [
        "@agenson-horrowitz/web-content-extractor-mcp"
      ]
    }
  }
}

Tools

No tools detected

We auto-extract tools from the README. The maintainer can list them under a ## Tools heading to populate this section.

Overview

What is Web Content Extractor (agent Optimized)?

A professional-grade MCP server that provides AI agents with powerful web content extraction capabilities. It extracts clean, structured, LLM-optimized content from web pages, saving tokens and improving agent accuracy by converting raw HTML into markdown, JSON, or structured data.

How to use Web Content Extractor (agent Optimized)?

Install via npm globally (npm install -g @agenson-horrowitz/web-content-extractor-mcp) or configure it in Claude Desktop or Cline by adding the server to the respective MCP config JSON with command npx and args ["@agenson-horrowitz/web-content-extractor-mcp"]. Once configured, users invoke specific tools such as extract_article, extract_structured_data, extract_links, screenshot_to_markdown, or batch_extract.

Key features of Web Content Extractor (agent Optimized)

Advanced article extraction with clean markdown and metadata
Structured data parsing (tables, lists, forms) as JSON
Intelligent link analysis with categorization and context
Visual layout analysis via screenshot-to-markdown
High-performance batch processing with rate limiting
Sub-2-second response times and token-efficient output

Use cases of Web Content Extractor (agent Optimized)

AI agent reads and summarizes news articles, blog posts, or research papers
Extract pricing tables and feature comparisons for competitive analysis
Perform bulk content audits across multiple competitor websites
Analyze UI layouts and visual content for design understanding
Discover and categorize internal/external links for site mapping or SEO

FAQ from Web Content Extractor (agent Optimized)

How does Web Content Extractor (agent Optimized) differ from raw HTML scraping?

It extracts LLM-optimized content with structured metadata, saving tokens and improving accuracy compared to raw HTML. Uses Mozilla Readability, Playwright, and other libraries for clean output.

What are the runtime requirements and dependencies?

Requires Node.js; uses Playwright for browser automation, Mozilla Readability for content extraction, Metascraper for metadata, Turndown for HTML-to-markdown, and JSDOM for DOM manipulation.

Where does the extracted data live?

All data is extracted from provided URLs and returned in the tool response; no persistent storage on the server side is mentioned.

What are the known limits of Web Content Extractor (agent Optimized)?

Average response time < 2 seconds; rate limit of 10 extractions/second (configurable); content limit of 50MB per extraction; free tier allows 500 extractions/month, with higher limits on paid plans.

What transport and authentication options are available?

Uses MCP protocol via stdio transport (default setup via npx). Authentication is not required for local usage; for hosted usage via MCPize or direct API, it supports API keys and crypto micropayments (USDC on Base chain).