Inferbench

Name: Inferbench
Author: JoniMartin27

@JoniMartin27

关于 Inferbench

InferBench's MCP server lets coding agents run, serve and benchmark local LLMs (text + image, llama.cpp + Stable Diffusion) on your own hardware on demand. Measures real tokens/sec, picks the optimal quant for your GPU, and exposes a 124-model catalog. Local-first, no cloud requi

#llm #local-llm #benchmark #llama-cpp #inference #image-generation

基本信息

分类

其他

传输方式

stdio

发布者

JoniMartin27

提交者

Jonathan Martin Paez

配置

使用下面的配置,将此服务器添加到你的 MCP 客户端。

{
  "mcpServers": {
    "inferbench": {
      "command": "C:\\Users\\<user>\\AppData\\Local\\Programs\\InferBench\\resources\\sidecar\\inferbench-backend.exe",
      "args": [
        "--mcp"
      ]
    }
  }
}

工具

未检测到工具

工具是从 README 中自动提取的。维护者可以在 ## Tools 标题下列出工具,即可填充这部分内容。

概览

What is Inferbench?

Inferbench is an MCP server that lets coding agents run, serve, and benchmark local LLMs (text + image, via llama.cpp and Stable Diffusion) on your own hardware. It measures real tokens/second, picks the optimal quantization for your GPU, and exposes a 124-model catalog.

How to use Inferbench?

—

Key features of Inferbench

Run, serve, and benchmark local LLMs on demand
Measures real tokens per second throughput
Selects optimal quantization for your GPU automatically
Supports text and image models (llama.cpp + Stable Diffusion)
Exposes a catalog of 124 models
Operates entirely local-first, no cloud required

Use cases of Inferbench

Benchmarking local LLM performance across different quantizations and hardware
Running coding agent tasks against locally served text and image models
Identifying the best-performing model quantization for a given GPU

FAQ from Inferbench

What models does Inferbench support?

Inferbench supports text models via llama.cpp and image models via Stable Diffusion, with a catalog of 124 models.

Where does data stay and is the cloud required?

Inferbench is local-first and requires no cloud.

What performance metrics does Inferbench measure?

It measures real tokens per second throughput.

其他分类下的更多 MCP 服务器

Reactive Resume

amruthpillai

A one-of-a-kind resume builder that keeps your privacy in mind. Completely secure, customizable, portable, open-source and free forever. Try it out today!

39.5K