Conkurrence MCP Server

@AlligatorC0der

リポジトリ

基本情報

カテゴリ

その他

トランスポート

stdio

作者

AlligatorC0der

投稿者

joe Etherage

Conkurrence MCP Server について

Conkurrence measures whether multiple AI models produce consistent outputs on your evaluation tasks. It tells you which items your AI agrees on and which need human review — using Fleiss' κ, Kendall's W, and bootstrap confidence intervals, the same psychometric methods trusted in

#ai-evaluation#golden-dataset#llm-evaluation #testing#inter-rater-reliability#fleiss-kappa#kendall-w#validating-prompt-output#consensus #multi-model

Conkurrence MCP Server の設定

以下の設定を使って、このサーバーを MCP 対応クライアントに追加してください。

{
  "mcpServers": {
    "conkurrence": {
      "command": "npx",
      "args": [
        "-y",
        "conkurrence",
        "mcp"
      ]
    }
  }
}

Conkurrence MCP Server のツール

ツール一覧はまだありません

この掲載には表示できるツール一覧がまだありません。サーバーにツールがないという意味ではありません。対応機能はプロジェクトのドキュメントをご確認ください。

上のボタンで一時的なサンドボックスでこのサーバーを実行し、ツール一覧を取得できます。

概要

What is Conkurrence?

ConKurrence is a statistically validated consensus measurement toolkit for AI evaluation pipelines. It uses multiple AI models as independent raters, measures inter-rater reliability with Fleiss' kappa and bootstrap confidence intervals, and routes contested items to human experts.

How to use Conkurrence?

Install globally with npm install -g conkurrence, then run as an MCP server with npx conkurrence mcp. Add the corresponding configuration to your Claude Desktop or Claude Code plugin settings. Invoke the available MCP tools such as conkurrence_run, conkurrence_report, conkurrence_compare, conkurrence_trend, conkurrence_suggest, conkurrence_validate_schema, or conkurrence_estimate.

Key features of Conkurrence

Multi-model evaluation using Bedrock, OpenAI, and Gemini simultaneously
Fleiss' kappa with bootstrap confidence intervals and Kendall's W
Self-consistency mode (no API keys needed, uses MCP Sampling)
AI-powered schema suggestion from your data
Trend tracking to detect agreement degradation over time
Cost estimation before running an evaluation

Use cases of Conkurrence

Evaluate inter-rater reliability when multiple AI models judge the same data
Automatically route low-agreement items to human experts for review
Track agreement degradation across evaluation runs over time
Estimate and control costs before executing a multi-model evaluation
Design consistent evaluation schemas with AI-assisted suggestions

よくある質問

What problem does Conkurrence solve?

It provides a statistically validated consensus measurement for AI evaluation pipelines by using multiple models as independent raters.

Does Conkurrence require API keys for every run?

No—the self-consistency mode uses the host model via MCP Sampling and does not require any API keys.

What statistical measures does Conkurrence use?

It uses Fleiss' kappa with bootstrap confidence intervals and Kendall's W for inter-rater reliability and validity.

Can I estimate costs before running an evaluation?

Yes, the `conkurrence_estimate` tool reports the cost and token usage before execution.

What license does Conkurrence use?

It is licensed under BUSL-1.1 (Business Source License 1.1).

「その他」の他のコンテンツ

The Living Bread

A free remote MCP server for The Living Bread. Bible passages read word for word from a stored public domain text with an evidence label, keyword search and cross references, real churches, gatherings and prayer groups w

Yuppsy

Yuppsy MCP: build and publish real estate websites from your AI assistant. Yuppsy lets your AI assistant create live websites for real estate agents and individual property listings. You don't need a designer or any code