Where On-Device and Cloud LLMs Meet

What is Minions?

Minions is a communication protocol that enables small on-device language models to collaborate with frontier cloud models. By only reading long contexts locally, it reduces cloud costs with minimal or no quality degradation. This repository provides a Python package and demonstration applications, tested on Mac and Ubuntu with Python 3.10–3.11.

How to use Minions?

Clone the repository, install the Python package, and install a local model server (ollama or tokasaurus). Set API keys for at least one supported cloud provider (OpenAI, Together, DeepSeek, Anthropic, Mistral, Perplexity, OpenRouter). Then run the demo application with streamlit run app.py or follow example code in the README to define a Minion or Minions protocol with local and remote clients.

Key features of Minions

• Enables cost-efficient collaboration between on-device and cloud LLMs.
• Processes long contexts locally, reducing cloud inference costs.
• Supports multiple local servers: ollama, tokasaurus, llama-cpp-python, and WebGPU.
• Integrates with many cloud providers (OpenAI, Together, DeepSeek, Anthropic, etc.).
• Includes Streamlit and WebGPU demo applications.
• Offers Secure Minions Chat for end-to-end encrypted communication.

Use cases of Minions

• Analyzing long medical records with local and cloud LLMs to assess cardiovascular risk.
• Cost-efficient processing of large documents by keeping context on-device.
• Privacy-preserving AI tasks where sensitive data never leaves the local machine.

FAQ from Minions

What operating systems are supported?

Minions has been tested on Mac and Ubuntu with Python 3.10–3.11 (Python 3.13 is not supported).

What local model servers can I use?

You can use ollama, tokasaurus, or llama-cpp-python for local inference. A WebGPU app also allows browser-based local inference.

Which cloud LLM providers are supported?

Supported providers include OpenAI, Together AI, DeepSeek, Anthropic, Mistral AI, Perplexity, OpenRouter,