avaScript rendering time in ms (default: `2000`) ### Configuration Examples For standard usage: ```bash # Required export WEBSCRAPING_AI_API_KEY=your-api-key # Optional - customize behavior (default values) export WEBSCRAPING_AI_CONCURRENCY_LIMIT=5 export WEBSCRAPING_AI_DEFAULT_PROXY_TYPE=residential # datacenter or residential export WEBSCRAPING_AI_DEFAULT_JS_RENDERING=true export WEBSCRAPING_AI_DEFAULT_TIMEOUT=15000 export WEBSCRAPING_AI_DEFAULT_JS_TIMEOUT=2000 ``` ## Available Tools ### 1. Question Tool (`webscraping_ai_question`) Ask questions about web page content. ```json { "name": "webscraping_ai_question", "arguments": { "url": "https://example.com", "question": "What is the main topic of this page?", "timeout": 30000, "js": true, "js_timeout": 2000, "wait_for": ".content-loaded", "proxy": "datacenter", "country": "us" } } ``` Example response: ```json { "content": [ { "type": "text", "text": "The main topic of this page is examples and documentation for HTML and web standards." } ], "isError": false } ``` ### 2. Fields Tool (`webscraping_ai_fields`) Extract structured data from web pages based on instructions. ```json { "name": "webscraping_ai_fields", "arguments": { "url": "https://example.com/product", "fields": { "title": "Extract the product title", "price": "Extract the product price", "description": "Extract the product description" }, "js": true, "timeout": 30000 } } ``` Example response: ```json { "content": [ { "type": "text", "text": { "title": "Example Product", "price": "$99.99", "description": "This is an example product description." } } ], "isError": false } ``` ### 3. HTML Tool (`webscraping_ai_html`) Get the full HTML of a web page with JavaScript rendering. ```json { "name": "webscraping_ai_html", "arguments": { "url": "https://example.com", "js": true, "timeout": 30000, "wait_for": "#content-loaded" } } ``` Example response: ```json { "content": [ { "type": "text", "text": "...[full HTML content]..." } ], "isError": false } ``` ### 4. Text Tool (`webscraping_ai_text`) Extract the visible text content from a web page. ```json { "name": "webscraping_ai_text", "arguments": { "url": "https://example.com", "js": true, "timeout": 30000 } } ``` Example response: ```json { "content": [ { "type": "text", "text": "Example Domain\nThis domain is for use in illustrative examples in documents..." } ], "isError": false } ``` ### 5. Selected Tool (`webscraping_ai_selected`) Extract content from a specific element using a CSS selector. ```json { "name": "webscraping_ai_selected", "arguments": { "url": "https://example.com", "selector": "div.main-content", "js": true, "timeout": 30000 } } ``` Example response: ```json { "content": [ { "type": "text", "text": "

This is the main content of the page.

" } ], "isError": false } ``` ### 6. Selected Multiple Tool (`webscraping_ai_selected_multiple`) Extract content from multiple elements using CSS selectors. ```json { "name": "webscraping_ai_selected_multiple", "arguments": { "url": "https://example.com", "selectors": ["div.header", "div.product-list", "div.footer"], "js": true, "timeout": 30000 } } ``` Example response: ```json { "content": [ { "type": "text", "text": [ "", "

Product list content

", "" ] } ], "isError": false } ``` ### 7. Account Tool (`webscraping_ai_account`) Get information about your WebScraping.AI account. ```json { "name": "webscraping_ai_account", "arguments": {} } ``` Example response: ```json { "content": [ { "type": "text", "text": { "req

WebScraping.AI MCP Server

Comments