Overview
What is Mcp_web_scrapper?
Mcp_web_scrapper is a Node.js tool that scrapes and extracts MCP web server listings from glama.ai. It is intended for developers who want to collect a directory of available MCP servers.
How to use Mcp_web_scrapper?
Clone the repository, run npm install, then execute node scraper.js to collect server links, followed by node process-links.js to extract detailed data. Delete the servers-json folder if a fresh scrape is needed.
Key features of Mcp_web_scrapper
- Scrapes MCP server links from glama.ai
- Extracts detailed server information from each link
- Uses a headless browser for data collection
- Saves output as JSON files in a dedicated folder
- Provides terminal progress during processing
Use cases of Mcp_web_scrapper
- Building a local directory of MCP servers
- Analyzing the MCP server ecosystem from glama.ai
- Automating the discovery of new MCP servers
- Creating a backup of server listings for offline use
FAQ from Mcp_web_scrapper
What does Mcp_web_scrapper do?
It scrapes and extracts MCP web servers from the glama.ai website, first collecting all server links and then fetching detailed data from each link.
How do I install Mcp_web_scrapper?
Clone the repository and run npm install to install dependencies. You need Node.js and npm.
How do I run a fresh scrape?
Delete the servers-json folder if it exists from a previous run, then execute node scraper.js followed by node process-links.js.
What files are generated?
A file named server-links.json is created in a servers-json folder. After processing, detailed data files named after each MCP server are stored in the same folder.
How long does scraping take?
The process may take a few minutes. Wait for the “Closing browser” message before terminating.
