Root Signals Mcp

Name: Root Signals Mcp
Author: root-signals

@root-signals

Repository Homepage

About Root Signals Mcp

MCP for Scorable Evaluation Platform

#evals #llm-as-a-judge

Basic information

Config

Add this server to your MCP-compatible client using the configuration below.

{
  "mcpServers": {
    "root-signals-mcp": {
      "command": "docker",
      "args": [
        "run",
        "-e",
        "SCORABLE_API_KEY=<your_key>",
        "-p",
        "0.0.0.0:9090:9090",
        "--name=rs-mcp",
        "-d",
        "ghcr.io/scorable/scorable-mcp:latest"
      ]
    }
  }
}

Tools

No tools detected

We auto-extract tools from the README. The maintainer can list them under a ## Tools heading to populate this section.

Overview

What is Scorable MCP?

Scorable MCP is a Model Context Protocol (MCP) server that bridges the Scorable API with MCP client applications. It exposes Scorable evaluators as tools for AI assistants and agents, enabling automated evaluation of responses against various quality criteria.

How to use Scorable MCP?

Get an API key from Scorable, then run the server using Docker with SSE transport (docker run -e SCORABLE_API_KEY=<key> -p 9090:9090 ghcr.io/scorable/scorable-mcp:latest) or via stdio using uvx. Configure the server in your MCP client (e.g., Cursor) by pointing to the SSE endpoint or adding the stdio command with the API key.

Key features of Scorable MCP

Exposes Scorable evaluators as MCP tools
Supports SSE transport for network deployment
Compatible with MCP clients like Cursor
Tools for listing and running evaluations
Supports evaluation by name or ID
Includes a reference MCP client for Python

Use cases of Scorable MCP

Evaluate and improve AI agent explanations automatically.
Measure prompt templates for clarity and precision.
Run coding policy adherence evaluations using policy documents.
Use a combination of judges (collections of evaluators) for comprehensive assessment.

FAQ from Scorable MCP

How do I get an API key for Scorable MCP?

What transport methods does Scorable MCP support?

Scorable MCP supports SSE transport (via Docker, port 9090) and stdio transport (via uvx).

What are the main limitations of Scorable MCP?

The current implementation lacks exponential backoff, automatic retries, and request throttling. The bundled MCP client is for reference only and has no support guarantees.

Can I use Scorable MCP with Cursor?

Yes, the README provides example configurations for Cursor, using either SSE or stdio transport.

What kinds of evaluations can I run?

You can run standard evaluations by evaluator ID or name, coding policy adherence evaluations, and judges (collections of evaluators). Additional tools list all available evaluators and judges on your account.