What problem does Conkurrence solve?
It provides a statistically validated consensus measurement for AI evaluation pipelines by using multiple models as independent raters.
C
Conkurrence MCP Server
@AlligatorC0der
Basic information
About Conkurrence MCP Server
Conkurrence measures whether multiple AI models produce consistent outputs on your evaluation tasks. It tells you which items your AI agrees on and which need human review — using Fleiss' κ, Kendall's W, and bootstrap confidence intervals, the same psychometric methods trusted in
#ai-evaluation#golden-dataset#llm-evaluation#testing#inter-rater-reliability#fleiss-kappa#kendall-w#validating-prompt-output#consensus#multi-model
Conkurrence MCP Server config
Add this server to your MCP-compatible client using the configuration below.
{
"mcpServers": {
"conkurrence": {
"command": "npx",
"args": [
"-y",
"conkurrence",
"mcp"
]
}
}
}Conkurrence MCP Server tools
Tool list not available yet
This listing does not currently have a tool list to display. This does not mean the server has no tools; check its documentation for supported capabilities.
Fetch the live tool list by running this server in a temporary sandbox using the button above.
Overview
What is Conkurrence?
ConKurrence is a statistically validated consensus measurement toolkit for AI evaluation pipelines. It uses multiple AI models as independent raters, measures inter-rater reliability with Fleiss' kappa and bootstrap confidence intervals, and routes contested items to human experts.
How to use Conkurrence?
Install globally with npm install -g conkurrence, then run as an MCP server with npx conkurrence mcp. Add the corresponding configuration to your Claude Desktop or Claude Code plugin settings. Invoke the available MCP tools such as conkurrence_run, conkurrence_report, conkurrence_compare, conkurrence_trend, conkurrence_suggest, conkurrence_validate_schema, or conkurrence_estimate.
Key features of Conkurrence
- Multi-model evaluation using Bedrock, OpenAI, and Gemini simultaneously
- Fleiss' kappa with bootstrap confidence intervals and Kendall's W
- Self-consistency mode (no API keys needed, uses MCP Sampling)
- AI-powered schema suggestion from your data
- Trend tracking to detect agreement degradation over time
- Cost estimation before running an evaluation
Use cases of Conkurrence
- Evaluate inter-rater reliability when multiple AI models judge the same data
- Automatically route low-agreement items to human experts for review
- Track agreement degradation across evaluation runs over time
- Estimate and control costs before executing a multi-model evaluation
- Design consistent evaluation schemas with AI-assisted suggestions
Frequently asked questions
Does Conkurrence require API keys for every run?
No—the self-consistency mode uses the host model via MCP Sampling and does not require any API keys.
What statistical measures does Conkurrence use?
It uses Fleiss' kappa with bootstrap confidence intervals and Kendall's W for inter-rater reliability and validity.
Can I estimate costs before running an evaluation?
Yes, the `conkurrence_estimate` tool reports the cost and token usage before execution.
What license does Conkurrence use?
It is licensed under BUSL-1.1 (Business Source License 1.1).
More Other MCP servers
Blender
ahujasidOpen-source MCP to use Blender with any LLM
23.6K
Glasswarp
GlasswarpSee and control a real Windows PC you own — from any MCP client, locally or remotely. Observe (UIA + screenshots), click/type/drag/scroll, launch apps, owner Live View. BYOH: your machine, your key.
P
PickFu MCP
PickFu MCP is a Model Context Protocol (MCP) server that connects AI assistants, developers, marketers, product managers, and AI agents to PickFu's consumer research platform. It enables users to create surveys, conduct
MaxKB
1Panel-dev🔥 MaxKB is an open-source platform for building enterprise-grade agents. 强大易用的开源企业级智能体平台。
22.0K
Senso MCP for Joomla
sensomediaSenso MCP turns a Joomla 5 or 6 site into a remote MCP server. It runs inside Joomla itself (no extra server, no Node.js bridge) and is secured with OAuth 2.1 and Joomla's own ACL. The free pack lets an AI assistant crea
Comments