Swift MCP GUI Server

Name: Swift MCP GUI Server
Author: NakaokaRei

@NakaokaRei

Repository

About Swift MCP GUI Server

MCP server that can execute commands such as keyboard input and mouse movement on macOS

Basic information

Config

No standard config provided

This server doesn't expose a parseable MCP config block in its README. See the repository for install instructions.

Repository

Tools

No tools detected

We auto-extract tools from the README. The maintainer can list them under a ## Tools heading to populate this section.

Overview

What is Swift MCP GUI Server?

A Model Context Protocol (MCP) server that allows controlling macOS through SwiftAutoGUI. It provides tools for programmatically controlling the mouse and keyboard through MCP clients.

How to use Swift MCP GUI Server?

Install via Homebrew (brew install NakaokaRei/tap/swift-mcp-gui) or from source using swift package experimental-install. Then add the binary path to your MCP client configuration (e.g., /opt/homebrew/bin/swift-mcp-gui on Apple Silicon, /usr/local/bin/swift-mcp-gui on Intel, or ~/.swiftpm/bin/swift-mcp-gui for source builds).

Key features of Swift MCP GUI Server

Move mouse cursor to specified coordinates
Perform left or right mouse clicks
Send keyboard shortcuts and key combinations
Scroll in four directions with configurable clicks
Capture full screen or a region as file path or inline image
Execute AppleScript code directly or from a file

Use cases of Swift MCP GUI Server

Automate repetitive macOS GUI tasks via MCP clients
Provide AI agents with screen capture and mouse/keyboard control
Test UI interactions by programmatically clicking and typing
Retrieve pixel colors or screen dimensions for automation scripts
Execute AppleScripts as part of a larger MCP workflow

FAQ from Swift MCP GUI Server

What are the system requirements for Swift MCP GUI Server?

macOS 15.0 or later, Swift 6.0 or later, and Xcode 16.0 or later.

How do I install Swift MCP GUI Server?

You can install it via Homebrew (brew install NakaokaRei/tap/swift-mcp-gui) or build from source using swift package experimental-install. Upgrade via brew upgrade swift-mcp-gui.

Do I need special permissions to use Swift MCP GUI Server?

Yes, the server requires full accessibility permissions in System Preferences to control your mouse and keyboard.

What tools does Swift MCP GUI Server provide?

It offers 11 tools: moveMouse, mouseClick, sendKeys, scroll, getScreenSize, getPixelColor, captureScreen, captureRegion, saveScreenshot, executeAppleScript, and executeAppleScriptFile. Each has documented input parameters.

How does screen capture output work?

You can choose between two output formats: "path" (saves to a temporary file and returns the file path with dimensions, reducing token consumption) or "image" (returns inline image content for AI vision, e.g., Claude). Quality and scale are adjustable.