Advertise
MCP.so
Sign In

Crawlab MCP Server

@crawlab-team

Repository

Basic information

Category

Other

Transports

stdio

Author

crawlab-team

About Crawlab MCP Server

No overview available yet

#crawlab#mcp-server#ai-integration

Crawlab MCP Server config

Add this server to your MCP-compatible client using the configuration below.

{
  "mcpServers": {
    "crawlab-mcp-server": {
      "type": "http",
      "url": "http://localhost:8000"
    }
  }
}

Crawlab MCP Server tools

Tool list not available yet

This listing does not currently have a tool list to display. This does not mean the server has no tools; check its documentation for supported capabilities.

Fetch the live tool list directly from this server's endpoint using the button above.

Overview

What is Crawlab MCP Server?

The Crawlab MCP Server is a Model Context Protocol (MCP) server that enables AI applications to interact with Crawlab’s spider, task, and file management features through natural language. It bridges an MCP client (such as Claude Desktop) with the Crawlab API, allowing users to create, run, and manage web scraping workflows without manual API calls.

How to use Crawlab MCP Server?

Install as a Python package (pip install -e .) and run the server CLI (crawlab_mcp-mcp server), or run directly with python server.py after setting up a .env file with your Crawlab API base URL and token. A Docker image is also available. Connect an AI client to the server’s URL (e.g., http://localhost:8000) to start issuing natural language commands.

Key features of Crawlab MCP Server

  • MCP‑based communication between AI apps and Crawlab
  • Full spider lifecycle: create, read, update, delete
  • Task management: run, cancel, restart, view logs
  • File operations: list, read, write spider files
  • Supports multiple deployment options (local, Docker, pip)
  • Configuration via environment variables (.env)

Use cases of Crawlab MCP Server

  • Create and configure new spiders using natural language
  • Launch scraping tasks on demand and monitor progress
  • Retrieve and update spider code files through a chat interface
  • Cancel or restart tasks without leaving the AI application

Frequently asked questions

1

What is the Crawlab MCP Server?

It is a standard MCP server that translates AI application tool calls into Crawlab API requests, enabling natural‑language control of Crawlab’s spider, task, and file management features.

2

How do I install the Crawlab MCP Server?

You can install it as a Python package (`pip install -e .`) and run via CLI, or run `python server.py` directly. A Docker build is also provided.

3

What are the prerequisites to run the server?

Python 3.8+, a running Crawlab instance, and a valid Crawlab API token. The server connects to Crawlab via its API, so the instance must be accessible.

4

How is authentication handled?

The server reads the Crawlab API token from the `CRAWLAB_API_TOKEN` environment variable (set in `.env`) and uses it to authenticate all requests to the Crawlab API.

5

What transport does the MCP server use?

The server communicates over HTTP. Clients connect to its URL (e.g., `http://localhost:8000`), and tool calls are executed via RESTful API calls to the Crawlab backend.

Comments

More Other MCP servers

Blender

ahujasid

Open-source MCP to use Blender with any LLM

23.6K

Yuppsy

Yuppsy MCP: build and publish real estate websites from your AI assistant. Yuppsy lets your AI assistant create live websites for real estate agents and individual property listings. You don't need a designer or any code

Glasswarp

Glasswarp

See and control a real Windows PC you own — from any MCP client, locally or remotely. Observe (UIA + screenshots), click/type/drag/scroll, launch apps, owner Live View. BYOH: your machine, your key.

MaxKB

1Panel-dev

🔥 MaxKB is an open-source platform for building enterprise-grade agents. 强大易用的开源企业级智能体平台。

22.0K

MCP Toolbox for Databases

googleapis

MCP Toolbox for Databases is an open source MCP server for databases.

15.9K
Advertise