Advertise
MCP.so
Sign In
C

Crawl4ai Middleware MCP Server

@Frankieli123

Repository

Basic information

Category

Other

Transports

stdio

Author

Frankieli123

Submitted by

About Crawl4ai Middleware MCP Server

No overview available yet

#crawl4ai#web-scraping#middleware

Crawl4ai Middleware MCP Server config

Add this server to your MCP-compatible client using the configuration below.

{
  "mcpServers": {
    "crawl4ai": {
      "command": "python",
      "args": [
        "E:/APP/craw14ai-middleware/crawl4ai_server.py"
      ]
    }
  }
}

Crawl4ai Middleware MCP Server tools

Tool list not available yet

This listing does not currently have a tool list to display. This does not mean the server has no tools; check its documentation for supported capabilities.

We auto-extract tools from the README. The maintainer can list them under a ## Tools heading to populate this section.

Overview

What is Crawl4ai Middleware?

Crawl4ai Middleware is an MCP (Model Context Protocol) server that integrates crawl4ai's web scraping capabilities into MCP-compatible applications like Cursor IDE. It provides tools to create crawl tasks, retrieve results by task ID, list saved files, and read their contents.

How to use Crawl4ai Middleware?

Install dependencies with pip install mcp[cli] httpx python-dotenv, configure environment variables (API base URL and key), then add the server configuration to your MCP client (e.g., Cursor IDE's settings.json) pointing to the crawl4ai_server.py script.

Key features of Crawl4ai Middleware

  • Provides four MCP tools for web scraping workflows
  • Saves scraped content locally to avoid display size limits
  • Supports environment variable configuration for API and output directory
  • Allows retrieval of crawl results by task ID
  • Lists and reads saved result files
  • Works with Cursor IDE and other MCP-enabled applications

Use cases of Crawl4ai Middleware

  • Scrape a web page and retrieve its content directly in Cursor IDE
  • Manage multiple crawl tasks with persistent file storage
  • Review previously crawled results offline at any time
  • Integrate web scraping into AI agent workflows via MCP

Frequently asked questions

1

What tools does Crawl4ai Middleware provide?

It provides `create_crawl_task`, `get_crawl_result`, `list_saved_results`, and `read_saved_result`.

2

How are crawled files stored?

Files are saved in a `url` folder next to the server script, named `extracted_domain_taskID_timestamp.txt`.

3

What dependencies are required?

Install `mcp[cli]`, `httpx`, and `python-dotenv` via pip.

4

How do I configure the server?

Set environment variables `CRAWL4AI_API_BASE`, `CRAWL4AI_API_KEY`, optionally `OUTPUT_DIR` and `LOG_LEVEL`. Default API base is `http://192.168.31.12:11235` and key `sk-3180623`.

5

Can I use this with Cursor IDE?

Yes, add the server configuration to Cursor's `settings.json` with the command `python` and args pointing to the `crawl4ai_server.py` script.

Comments

More Other MCP servers

MCP Toolbox for Databases

googleapis

MCP Toolbox for Databases is an open source MCP server for databases.

15.9K

🚀 Model Context Protocol (MCP) Curriculum for Beginners

microsoft

This open-source curriculum introduces the fundamentals of Model Context Protocol (MCP) through real-world, cross-language examples in .NET, Java, TypeScript, JavaScript, Rust and Python. Designed for developers, it focuses on practical techniques for building modular, scalable,

16.7K

Yuppsy

Yuppsy MCP: build and publish real estate websites from your AI assistant. Yuppsy lets your AI assistant create live websites for real estate agents and individual property listings. You don't need a designer or any code

Glasswarp

Glasswarp

See and control a real Windows PC you own — from any MCP client, locally or remotely. Observe (UIA + screenshots), click/type/drag/scroll, launch apps, owner Live View. BYOH: your machine, your key.

Reactive Resume

amruthpillai

A one-of-a-kind resume builder that keeps your privacy in mind. Completely secure, customizable, portable, open-source and free forever. Try it out today!

39.5K
Advertise