Swift MCP GUI Server

What is Swift MCP GUI Server?

A Model Context Protocol (MCP) server that allows controlling macOS through SwiftAutoGUI. It provides tools for programmatically controlling the mouse and keyboard through MCP clients.

How to use Swift MCP GUI Server?

Install via Homebrew (brew install NakaokaRei/tap/swift-mcp-gui) or from source using swift package experimental-install. Then add the binary path to your MCP client configuration (e.g., /opt/homebrew/bin/swift-mcp-gui on Apple Silicon, /usr/local/bin/swift-mcp-gui on Intel, or ~/.swiftpm/bin/swift-mcp-gui for source builds).

Key features of Swift MCP GUI Server

• Move mouse cursor to specified coordinates
• Perform left or right mouse clicks
• Send keyboard shortcuts and key combinations
• Scroll in four directions with configurable clicks
• Capture full screen or a region as file path or inline image
• Execute AppleScript code directly or from a file

Use cases of Swift MCP GUI Server

• Automate repetitive macOS GUI tasks via MCP clients
• Provide AI agents with screen capture and mouse/keyboard control
• Test UI interactions by programmatically clicking and typing
• Retrieve pixel colors or screen dimensions for automation scripts
• Execute AppleScripts as part of a larger MCP workflow

FAQ from Swift MCP GUI Server

What are the system requirements for Swift MCP GUI Server?

macOS 15.0 or later, Swift 6.0 or later, and Xcode 16.0 or later.

How do I install Swift MCP GUI Server?

You can install it via Homebrew (brew install NakaokaRei/tap/swift-mcp-gui) or build from source using swift package experimental-install. Upgrade via brew upgrade swift-mcp-gui.

Do I need special permissions to use Swift MCP GUI Server?

Yes, the server requires full accessibility permissions in System Preferences to control your mouse and keyboard.

What tools does Swift MCP GUI Server provide?

It offers 11 tools: moveMouse, mouseClick, sendKeys, scroll, getScreenSize, getPixelColor, captureScreen, captureRegion, saveScreenshot, executeAppleScript, and executeAppleScriptFile. Each has documented input parameters.

How does screen capture output work?

You can choose between two output formats: "path" (saves to a temporary file and returns the file path with dimensions, reducing token consumption) or "image" (returns inline image content for AI vision, e.g., Claude). Quality and scale are adjustable.