Model Context Protocol (MCP) Agent Frameworks Demo

What is Model Context Protocol (MCP) Agent Frameworks Demo?

The Model Context Protocol (MCP) Agent Frameworks Demo is a demonstration and benchmarking platform that shows how to integrate Model Context Protocol (MCP) servers with multiple LLM agent frameworks: Google Agent Development Kit (ADK), LangGraph Agents, OpenAI Agents, and Pydantic‑AI Agents. It includes example Python MCP servers and a comprehensive evaluation suite for benchmarking LLM performance on mermaid diagram correction tasks.

How to use Model Context Protocol (MCP) Agent Frameworks Demo?

Set environment variables (GEMINI_API_KEY and/or OPENAI_API_KEY) in a .env file, then run one of the example scripts, e.g., uv run agents_mcp_usage/basic_mcp/basic_mcp_use/pydantic_mcp.py. For benchmarking, use make benchmark MODEL="..." RUNS=5 PARALLEL=1 or the multi‑model evaluation scripts. Optional LOGFIRE_TOKEN enables tracing visualisation in the Logfire web UI.

Key features of Model Context Protocol (MCP) Agent Frameworks Demo

• Demonstrates MCP integration with four agent frameworks
• Includes example Python MCP servers (example_server.py, mermaid_validator.py)
• Supports single and multiple MCP server coordination
• Provides comprehensive evaluation and benchmarking suite
• Offers interactive Streamlit dashboard for model comparison
• Uses Pydantic Logfire for tracing and debugging

Use cases of Model Context Protocol (MCP) Agent Frameworks Demo

• Learning how to connect MCP servers to different agent frameworks
• Benchmarking LLM performance on mermaid diagram correction tasks
• Evaluating cost, accuracy, and speed across multiple models
• Developing modular, scalable agent architectures with multiple MCP servers

FAQ from Model Context Protocol (MCP) Agent Frameworks Demo

What is MCP?

The Model Context Protocol allows applications to provide context for LLMs in a standardised way, separating the concerns of providing context from the actual LLM interaction.

Which agent frameworks are demonstrated?

Google Agent Development Kit (ADK), LangGraph Agents, OpenAI Agents, and Pydantic‑AI Agents are demonstrated.

What MCP servers are included in the repository?

Two Python MCP servers: example_server.py (provides an add(a,b) tool, get_current_time() tool, and a dynamic greeting resource) and mermaid_validator.py (Mermaid diagram validation using mermaid‑cli).

How do I run a single‑model benchmark?

Use the Makefile command: make benchmark MODEL="openai:gpt-5.2 (none)" RUNS=5 PARALLEL=1 to run a Dockerised single‑model benchmark with isolated dependencies.

What tracing tool is used?

Tracing is done through Pydantic Logfire. An optional LOGFIRE_TOKEN can be set to visualise evaluations in the Logfire web UI.