Llama Hot Swap

What is Llama Hot Swap?

Llama Hot Swap is an MCP server that hot-swaps llama.cpp models inside a running Claude Code session, using launchctl (macOS) or systemd (Linux) to manage model services. It preserves full conversation context by keeping the message history client-side, enabling seamless transitions between reasoning and coding models on local hardware. It is designed for developers running local LLMs who need to switch models without losing context or using cloud APIs.

How to use Llama Hot Swap?

Install with pip install mcp-llama-swap or run directly with uvx mcp-llama-swap. Configure the MCP server in ~/.claude.json with the LLAMA_SWAP_CONFIG environment variable pointing to a config.json that maps model aliases to service files. Then inside Claude Code use commands like “list models” and “swap to planner” or “swap to coder”.

Key features of Llama Hot Swap

• Hot-swap llama.cpp models without context loss
• Supports macOS (launchctl) and Linux (systemd)
• Mapped mode and directory mode for model discovery
• Create new model service configs from within Claude Code
• Works with any llama.cpp‑compatible model (GGUF)

Use cases of Llama Hot Swap

• Plan architecture with a reasoning model, then swap to a coding model to implement—all in the same session.
• Rapidly iterate across models without restarting or losing conversation history.
• Run entirely offline on local hardware with no data leaving your machine.

FAQ from Llama Hot Swap

What does Llama Hot Swap do that manually swapping models cannot?

Manually swapping models would kill the conversation context and require restarting. Llama Hot Swap keeps context client‑side and only swaps the model behind the server, so the full history is preserved.

What are the runtime requirements?

Python 3.10+, macOS with launchctl or Linux with systemd, llama-server (llama.cpp) installed, and a LiteLLM proxy running to translate Anthropic‑format requests to OpenAI‑format for llama-server.

Where does conversation data live?

Conversation history is stored entirely on the client side (Claude Code) and re‑sent with every request. No data is persisted or leaves your machine.

Are there known limitations?

Large models may need more than the default 30‑second health timeout; increase health_timeout in the config. Also, only one model can be loaded at a time because the machine cannot hold multiple large models simultaneously.

What transports and authentication does Llama Hot Swap use?

As an MCP server, it communicates over standard I/O with the Claude Code client. No authentication is configured; the server manages local service files and health checks against the local llama-server instance.