MSPaint MCP Server with AI-based Planning Algorithms
@shettysaish20
Using Advanced AI Prompting to enhance LLM planning to solve complex math problems and draw the answer on MSPaint Canvas
Overview
What is MSPaint MCP Server with AI-based Planning Algorithms?
An MCP (Model Context Protocol) server that enables an AI agent powered by Google Gemini to automate legacy Windows MSPaint. The agent solves complex multi‑step math problems and draws the final answer on the Paint canvas using tools defined via fastmcp and pywinauto. It is intended for developers and researchers exploring structured prompting and tool‑augmented AI workflows.
How to use MSPaint MCP Server with AI-based Planning Algorithms?
- Create a Conda environment with Python 3.11 and activate it.
- Install dependencies with
pip install -r requirements.txt. - Place your Gemini API key in a
.envfile asGEMINI_API_KEY=YOUR_KEY. - Run the client:
python mcp_paint_app/mcp_client.py.
This starts the client, which connects to the MCP server, initializes the AI agent, and begins solving the provided math expression.
FAQ from MSPaint MCP Server with AI-based Planning Algorithms
What are the system dependencies and runtime requirements?
Python 3.11+ and Conda are recommended. Dependencies include pywin32, pywinauto, fastmcp, python-dotenv, google-genai, and rich. A Google Gemini API key is required.
What if the AI agent does not select the correct tools?
Review the system prompt in the client code; ensure tool descriptions are accurate and complete. Adjust the prompt if necessary.
How do I fix coordinate issues when drawing in MSPaint?
Coordinates used for clicking and drawing may need adjustment based on your screen resolution and window size. Use the debugging print statements in the code to identify and set the correct values.
What should I do if I encounter permission errors?
Try running
