Ingero

Name: Ingero
Author: ingero-io

@ingero-io

About Ingero

eBPF-based GPU causal observability agent with MCP server. Traces CUDA Runtime/Driver APIs and host kernel events to build causal chains explaining GPU latency

#ebpf #gpu #cuda #observability #monitoring #mcp #pytorch #kubernetes #tracing

Basic information

Config

Add this server to your MCP-compatible client using the configuration below.

{
  "mcpServers": {
    "ingero": {
      "command": "ingero",
      "args": [
        "mcp"
      ],
      "env": {}
    }
  }
}

Tools

No tools detected

We auto-extract tools from the README. The maintainer can list them under a ## Tools heading to populate this section.

Overview

What is Ingero?

Ingero is an eBPF-based agent that provides production-safe, kernel-level causal tracing for GPU workloads. It answers the question: "Why is my GPU training/inference slow right now?" It is designed for developers and operators debugging GPU performance issues.

How to use Ingero?

Install the single Ingero binary, then start the MCP server with ingero mcp --db ~/.ingero/ingero.db for stdio transport (Claude Code, Cursor) or ingero mcp --db ~/.ingero/ingero.db --http :8090 for HTTPS. No code changes are required. Use the provided MCP tools to diagnose performance.

Key features of Ingero

eBPF-based causal tracing across 4 layers (CUDA API, host kernel, system context)
Production-safe with <2% overhead and zero code changes required
Python source line attribution (3.10/3.11/3.12) via DWARF
SQLite storage with 10 GB rolling cap
Kubernetes support (DaemonSet, Helm chart, pod metadata)
Single binary, no runtime dependencies

Use cases of Ingero

Identifying the root cause of slow GPU training or inference
Debugging CUDA kernel launch latencies and memory bottlenecks
Analyzing system-level events (CPU scheduling, OOM, memory pressure) affecting GPU workloads
Running synthetic demos to test instrumentation without a GPU or root access

FAQ from Ingero

What data does Ingero collect and where is it stored?

Ingero traces CUDA Runtime and Driver API calls, host kernel tracepoints, and system context from /proc. All event data is stored in a local SQLite database at the configured path (default ~/.ingero/ingero.db) with a 10 GB rolling cap.

Does Ingero require code changes or root access?

No code changes are needed. For production monitoring, root access is required to attach eBPF programs; however, the run_demo tool can be used without a GPU or root privileges.

What GPU hardware and environments are supported?

Ingero is tested on A10, A100, H100, GH200, RTX 3090, and RTX 4090. It supports Kubernetes deployments via a DaemonSet and Helm chart, and it works with Python 3.10/3.11/3.12.

What transport protocols and authentication does Ingero use?

Ingero's MCP server supports stdio (local) and HTTP (HTTPS for remote clients). The README does not specify built-in authentication mechanisms.

How does Ingero differ from other GPU profiling tools?

Ingero provides causal chains that link system context and host-level events to specific CUDA calls, offering root cause analysis with severity ranking and fix recommendations, rather than just aggregate metrics or sampling profiles.