MCP Smart Crawler

Name: MCP Smart Crawler
Author: loo-y

@loo-y

About MCP Smart Crawler

MCP Smart Crawler is a Model Context Protocol (MCP) server uses Playwright to crawl web content, extract metadata, and download resources such as videos and images.

Basic information

Config

Add this server to your MCP-compatible client using the configuration below.

{
  "mcpServers": {
    "mcp-smart-crawler": {
      "command": "npx",
      "args": [
        "-y",
        "mcp-smart-crawler@latest",
        "--download-folder",
        "C:\\downloads"
      ]
    }
  }
}

Tools

No tools detected

We auto-extract tools from the README. The maintainer can list them under a ## Tools heading to populate this section.

Overview

What is MCP Smart Crawler?

MCP Smart Crawler is a Model Context Protocol (MCP) server that uses Playwright to crawl web content, extract metadata, and download resources such as videos and images. It is designed for users who need to programmatically retrieve and save content from Xiaohongshu (小红书) share links.

How to use MCP Smart Crawler?

Configure your MCP client with the provided JSON, setting the command to npx and adding optional arguments like --download-folder to specify a local download directory. Then send a chat message containing a Xiaohongshu share link to trigger crawling and downloading.

Key features of MCP Smart Crawler

Extracts metadata (title, description, images) from Xiaohongshu posts.
Downloads videos and images from Xiaohongshu share links.
Uses Playwright for browser automation.
Configurable download folder via command-line argument.

Use cases of MCP Smart Crawler

Automatically fetching and saving Xiaohongshu post images for content aggregation.
Downloading videos from shared Xiaohongshu links for offline viewing.
Extracting metadata from Xiaohongshu posts for analysis or archiving.