omniparser-autogui-mcp

What is omniparser-autogui-mcp?

omniparser-autogui-mcp is an MCP server that uses Microsoft's OmniParser to analyze the computer screen and automatically perform GUI operations. It is confirmed to work on Windows and is intended for users who want AI-driven desktop automation.

How to use omniparser-autogui-mcp?

Clone the repository recursively, run uv sync, set the OCR_LANG environment variable (e.g., en), and execute uv run download_models.py to download OmniParser models. Then configure the server in claude_desktop_config.json with the uv command pointing to the cloned directory. Additional environment variables allow customization of the backend load, target window, OmniParser server address, SSE transport, and model paths.

Key features of omniparser-autogui-mcp

• Integrates OmniParser for screen analysis and GUI automation
• Supports both stdio and SSE transport modes
• Allows targeting a specific window or the entire screen
• Can offload OmniParser processing to another device
• Configurable via environment variables for OmniParser model settings

Use cases of omniparser-autogui-mcp

• Automate repetitive desktop tasks based on screen content
• Search for specific text in a browser window and click results
• Operate legacy applications that lack APIs or scripting support
• Build AI assistants that can control the user interface directly

FAQ from omniparser-autogui-mcp

What is OmniParser?

OmniParser is a Microsoft research project that parses a screenshot into structured elements (e.g., icons, text boxes) for GUI automation. This server uses it under a CC-BY-4.0 license (models have separate licenses).

Does omniparser-autogui-mcp require a GPU?

The README does not specify GPU requirements. It is confirmed on Windows, and models are downloaded locally; performance may depend on the platform.

How does omniparser-autogui-mcp handle authentication or security?

No authentication mechanism is mentioned. The server operates on the local machine or over a network via SSE; no built-in security features are described.

Can I run OmniParser on a separate machine?

Yes. Set the OMNI_PARSER_SERVER environment variable to the server's address and port (e.g., 127.0.0.1:8000). Start the remote server with uv run omniparserserver.

What are the transport options?

The default transport is stdio. To use SSE, set the SSE_HOST and SSE_PORT environment variables.