2344 MCP Server

What is 2344?

An open-source platform for evaluating, testing, and monitoring LLM applications, built by Comet. It helps build LLM systems that run better, faster, and cheaper with tracing, evaluations, and dashboards.

How to use 2344?

Install the Python SDK with pip install opik, then run opik configure to set the local server address or a cloud API key. Alternatively, self-host with Docker Compose by cloning the repository and running ./opik.sh (Linux/Mac) or .\opik.ps1 (Windows). Once configured, log traces using the Python SDK or integrations.

Key features of 2344

• Trace all LLM calls and spans during development and production.
• Annotate traces with feedback scores via SDK or UI.
• Evaluate LLM applications with datasets, experiments, and LLM-as-a-judge metrics.
• Run evaluations as part of CI/CD pipelines with PyTest integration.
• Monitor production with high-volume ingestion (40M+ traces/day), dashboards, and online evaluation metrics.
• Support for OpenAI, LiteLLM, LangChain, and Haystack integrations.
• Prompt playground for experimenting with different prompts and models.

Use cases of 2344

• Development: trace, annotate, and experiment with LLM calls in real time.
• Evaluation: automate testing of LLM responses with custom datasets and judge metrics.
• Production monitoring: log high-volume traces, monitor feedback scores, and detect issues with online evaluation.
• CI/CD integration: validate LLM performance as part of automated testing pipelines.
• Self-hosted or cloud deployment: choose local control or fully managed Comet hosting.