Crawlio Browser
@Crawlio-app
About Crawlio Browser
100-tool browser automation for AI agents via MCP. Screenshots, DOM inspection, network capture, form filling, cookie management, tab orchestration, session recording, and structured data extraction via Chrome extension. One-command setup: npx crawlio-browser init auto-configures
Basic information
Config
Add this server to your MCP-compatible client using the configuration below.
{
"mcpServers": {
"crawlio-browser": {
"command": "npx",
"args": [
"-y",
"crawlio-browser"
]
}
}
}Tools
3Search available commands by keyword — both browser automation (via bridge.send) and Crawlio HTTP endpoints (via crawlio.api). Returns matching command names, descriptions, and parameter schemas. Use this to discover what commands are available before writing execute() code.
Execute JavaScript code with access to the browser bridge, Crawlio HTTP client, and smart object. Use search() first to discover available commands and their parameters. IMPORTANT WARNINGS: - smart.screenshot() does NOT exist. For screenshots: bridge.send({ type: 'take_screenshot' }). - For structured page evidence, prefer smart.extractPage() — runs 7 ops in parallel with typed gaps[]. - capture_page returns a ~1KB shaped summary. For raw data, use stop_network_capture or get_console_logs. - Use smart.waitForIdle() instead of sleep(). Use smart.scrollCapture() instead of manual scroll loops. - smart.snapshot() takes no options or { interactive: true } — there is no { compact: true } option. - For cross-page navigation, use smart.navigate(url) — never location.href = "..." (breaks CDP). Available in scope: - bridge.send(command, timeout?) — send command to browser extension via WebSocket command must have a `type` field matching a command name (e.g. { type: 'list_tabs' }) - crawlio.api(method, path, body?) — generic HTTP to ControlServer e.g. await crawlio.api('GET', '/status') e.g. await crawlio.api('POST', '/start', { url: 'https://example.com' }) e.g. await crawlio.api('POST', '/export', { format: 'zip', destinationPath: '/tmp/site.zip' }) e.g. await crawlio.api('PATCH', '/settings', { settings: { maxConcurrent: 8 } }) Returns { status: number, data: unknown } - crawlio.getStatus() — shortcut for GET /status - crawlio.startCrawl(url) — shortcut for POST /start - crawlio.getEnrichment(url?) — shortcut for GET /enrichment - crawlio.getCrawledURLs(params?) — shortcut for GET /crawled-urls - crawlio.postEnrichment(url, data) — shortcut for POST /enrichment/bundle - sleep(ms) — async wait (max 30s) - TIMEOUTS — per-command timeout constants - compileRecording(session, { name, description? }) — compile RecordingSession to SKILL.md Returns { skillMarkdown, name, pageCount, interactionCount } - ocrScreenshot(opts?) — extract text from current page via macOS Vision.framework OCR (macOS only) opts: { fullPage?: boolean, selector?: string } Returns { regions: [{ text, confidence, bounds }], regionsLimited? } - smart — auto-waiting wrappers and framework-specific data accessors: smart.evaluate(expr) → {result, type} — access .result for value. Never JSON.parse() the return directly. smart.click(selector, opts?) — poll + click + 500ms settle (accepts CSS or snapshot [ref=X]) smart.type(selector, text, opts?) — poll + type + 300ms settle smart.navigate(url, opts?) — navigate + 1000ms settle smart.waitFor(selector, timeout?) — poll until actionable smart.snapshot(opts?) — capture accessibility snapshot (opts: { interactive: true } for clickable elements only — NO compact option) smart.scrollCapture(opts?) — state-aware page scroll with screenshots, stops at page bottom smart.waitForIdle(timeout?) — wait for DOM mutations to settle (500ms quiet window) smart.extractPage(opts?) — capture_page + perf + security + fonts + meta + accessibility + mobileReadiness. Returns { capture, performance, security, fonts, meta, accessibility, mobileReadiness, gaps[] }. opts: { trace: true } adds _trace. smart.comparePages(urlA, urlB, opts?) — navigate to each URL, run extractPage(), return { siteA, siteB, scaffold }. scaffold has dimensions[], sharedFields, missingFields, metrics. smart.finding({ claim, evidence, sourceUrl, confidence, method, dimension? }) — create validated Finding, accumulate in session. Confidence auto-capped if dimension has active gap with reducesConfidence. smart.findings() — return all accumulated Finding[] from current session. smart.clearFindings() — reset accumulated findings and session gaps. smart.detectTables(opts?) — find repeating data patterns in the page. Returns TableCandidate[] (selector, score, rowCount, sampleText). Uses class-frequency scoring. smart.extractTable(selector, opts?) — extract structured data from a container. Returns { columns, rows, totalRows, truncated }. opts: { maxRows: 200 }. smart.waitForNetworkIdle(opts?) — wait for all network requests to settle (CDP-level, catches fetch/XHR/images/CSS/fonts). Returns { status, elapsed }. opts: { timeout: 15000, idleTime: 500 }. smart.extractData(opts?) — compound: detectTables + extractTable + JSON-LD. Returns { tables, structuredData, url }. smart.parseTrackingPixels() — parse captured network data for tracking pixel fires (Facebook, GA4, TikTok, LinkedIn, Pinterest). Returns { totalPixelFires, vendors, pixels, events, unrecognizedTrackingUrls }. smart.validateTracking() — validate tracking events against per-vendor parameter schemas (Facebook 18 standard + GA4 recommended). Returns { events, issues, errorCount, warningCount, infoCount, isHealthy }. Each issue has severity (error/warning/info), code, message, recommendation, and optional parameter. smart.inspectDataLayer() — inspect tracker runtime state via CDP (fbq queue, GA4 dataLayer, GTM containers, TikTok ttq). Returns DataLayerState with null for absent trackers. No content script needed. smart.detectDuplicates() — detect duplicate pixel fires grouped by vendor+pixelId+eventName+URL. Excludes PageView (legitimate SPA behavior). Returns DuplicateCluster[] with count and timestamps. smart.detectTechnologies(opts?) — detect technologies via fingerprint matching against CDP signals (headers, scripts, JS globals, meta, cookies, URL). Returns TechnographicResult { technologies[], categories, totalDetected, highConfidenceCount, signalsUsed }. Each technology has numeric confidence (0-100, additive), version, matchedSignals[]. opts: { confidenceThreshold: 1 }. smart.diffSnapshots(before?) — Myers diff current ARIA snapshot against baseline. If before omitted, uses last cached snapshot. Returns { diff, additions, removals, unchanged, changed }. Framework namespaces (injected based on detected framework): smart.react.{getVersion,getRootCount,hasProfiler,isHookInstalled} smart.vue.{getVersion,getAppCount,getConfig,isDevMode} smart.angular.{getVersion,isDebugMode,isIvy,getRootCount,getState} smart.svelte.{getVersion,getMeta,isDetected} smart.redux.{isInstalled,getStoreState} smart.alpine.{getVersion,getStoreKeys,getComponentCount} smart.nextjs.{getData,getRouter,getSSRMode,getRouteManifest} smart.nuxt.{getData,getConfig,isSSR} smart.remix.{getContext,getRouteData} smart.gatsby.{getData,getPageData} smart.shopify.{getShop,getCart} smart.wordpress.{isWP,getRestUrl,getPlugins} smart.laravel.{getCSRF} | smart.django.{getCSRF} | smart.drupal.{getSettings} smart.jquery.{getVersion} Example (HTTP API): const { data } = await crawlio.api('GET', '/status'); return data; Example (browser): const tabs = await bridge.send({ type: 'list_tabs' }, 5000); return tabs; Example (smart — auto-waiting click): await smart.click('#submit-btn'); return await smart.snapshot(); Example (smart — framework data): const nextData = await smart.nextjs?.getData(); return { page: nextData?.page, buildId: nextData?.buildId }; Example (session recording + compile): const s = await bridge.send({ type: 'start_recording', maxDurationSec: 120 }); // ... interact with page ... const session = await bridge.send({ type: 'stop_recording' }); const skill = compileRecording(session, { name: 'my-flow' }); return skill; IMPORTANT: Keep scripts fast (<15s). Each smart.click costs ~1-2s. Never loop 5+ clicks — use smart.evaluate to read DOM data in bulk instead. IMPORTANT: smart.evaluate returns {result, type}. Access .result for the value. Never JSON.stringify inside evaluate then JSON.parse outside — just return objects directly.
Pin a specific browser tab for all subsequent commands. Optional — without this, tools auto-connect to the active tab. Three modes: (1) provide a URL to find or create a tab, (2) provide a tabId to connect to a specific tab, (3) no args to pin the active tab. Starts CDP capture automatically.
Overview
What is Crawlio Browser?
Crawlio Browser is an MCP server that gives AI full control of a live Chrome browser via the Chrome DevTools Protocol (CDP), connecting through a lightweight Chrome extension rather than launching a separate headless browser. It is for developers and AI agents that need to interact with dynamic, JavaScript-rendered, or authenticated web content that static crawlers cannot handle.
How to use Crawlio Browser?
Install the Chrome extension, then run npx crawlio-browser init to auto-configure for 14 MCP clients. Alternatively, use manual configuration per client (e.g., Claude Desktop, Cursor, Windsurf). Transport modes include stdio (JSON‑RPC over stdin/stdout), Portal HTTP (Streamable HTTP), and Portal SSE (Server-Sent Events). Portal mode is recommended for Claude Code as it persists across session restarts.
Key features of Crawlio Browser
- 100 tools: 93 browser + 3 extraction + 3 recording + 1 compiler
- Framework-aware intelligence: detects 17 frameworks at runtime
- Evidence-based analysis with typed
CoverageGaprecords - Session recording and replay, compilable into SKILL.md automations
- Auto-settling: actionability checks and progressive backoff after mutations
- Tethered IPC bridge with WebSocket, heartbeat, and auto-reconnect
Use cases of Crawlio Browser
- AI agents that need to fill forms, click buttons, and navigate SPAs
- Automated testing of authenticated, framework-heavy web applications
- Extracting data from sites that rely on JavaScript rendering and dynamic state
- Recording browser interactions and compiling them into repeatable automation scripts
- Performing structured research with confidence-tracked findings and gap tracking
FAQ from Crawlio Browser
How does Crawlio Browser differ from headless browser tools?
Headless tools launch a separate browser process. Crawlio Browser connects to your existing Chrome via a lightweight extension, giving the AI access to your logged-in sessions, cookies, and full browser state — no separate browser, no login flows.
What runtime or dependencies are required?
You need Node.js and the Chrome browser with the Crawlio extension installed. The MCP server runs via npx crawlio-browser and communicates with the extension over WebSocket.
Where does the data live?
All browser interactions happen in your local Chrome session. No cloud processing is involved; the MCP server runs locally and communicates directly with your browser.
What transports and authentication are supported?
Three transports: stdio (default), Portal HTTP (Streamable HTTP), and Portal SSE. No authentication is described in the README; the server runs locally on your machine.
Are there any known limits?
The server has a message queue capacity of 100 messages and uses a heartbeat stale detection interval of 15 seconds. The JIT Context Runtime absorbs complexity from connection drops, tab refreshes, and port conflicts.
More Browser Automation MCP servers
Slack Slack
microsoftPlaywright is a framework for Web Testing and Automation. It allows testing Chromium, Firefox and WebKit with a single API.
X Twitter Scraper
Xquik-devTwitter scraper API skill for tweet search, advanced Twitter search, profile tweets, follower export, media download, monitors, webhooks, MCP, and posting automation: send tweets and replies.
Browser Control MCP
eyalzhMCP server paired with a browser extension that enables AI agents to control the user's browser.
Apify Model Context Protocol (MCP) Server
apifyThe Apify MCP server enables your AI agents to extract data from social media, search engines, maps, e-commerce sites, or any other website using thousands of ready-made scrapers, crawlers, and automation tools available on the Apify Store.
Scrapling Fetch MCP
cyberchittaHelps AI assistants access text content from bot-protected websites. MCP server that fetches HTML/markdown from sites with anti-automation measures using Scrapling.
Comments