Inferbench
@JoniMartin27
InferBench's MCP server lets coding agents run, serve and benchmark local LLMs (text + image, llama.cpp + Stable Diffusion) on your own hardware on demand. Measures real tokens/sec, picks the optimal quant for your GPU, and exposes a 124-model catalog. Local-first, no cloud requi
概览
What is Inferbench?
Inferbench is an MCP server that lets coding agents run, serve, and benchmark local LLMs (text + image, via llama.cpp and Stable Diffusion) on your own hardware. It measures real tokens/second, picks the optimal quantization for your GPU, and exposes a 124-model catalog.
How to use Inferbench?
—
Key features of Inferbench
- Run, serve, and benchmark local LLMs on demand
- Measures real tokens per second throughput
- Selects optimal quantization for your GPU automatically
- Supports text and image models (llama.cpp + Stable Diffusion)
- Exposes a catalog of 124 models
- Operates entirely local-first, no cloud required
Use cases of Inferbench
- Benchmarking local LLM performance across different quantizations and hardware
- Running coding agent tasks against locally served text and image models
- Identifying the best-performing model quantization for a given GPU
FAQ from Inferbench
What models does Inferbench support?
Inferbench supports text models via llama.cpp and image models via Stable Diffusion, with a catalog of 124 models.
Where does data stay and is the cloud required?
Inferbench is local-first and requires no cloud.
What performance metrics does Inferbench measure?
It measures real tokens per second throughput.
