Apache Beam MCP Server

@souravch

代码仓库

基本信息

分类

其他

许可证

MIT

传输方式

stdio

作者

souravch

关于 Apache Beam MCP Server

MCP server to manage apache beam workflows with different runners

Apache Beam MCP Server 配置

使用下面的配置,将此服务器添加到你的 MCP 客户端。

{
  "mcpServers": {
    "beam-mcp-server": {
      "command": "python",
      "args": [
        "-m",
        "venv",
        "beam-mcp-venv"
      ]
    }
  }
}

Apache Beam MCP Server 工具

工具列表暂未提供

当前收录暂时没有可展示的工具列表，不代表服务没有工具。请查阅项目文档了解支持的能力。

工具是从 README 中自动提取的。维护者可以在 ## Tools 标题下列出工具,即可填充这部分内容。

概览

What is Apache Beam MCP Server?

The Apache Beam MCP Server is a Model Context Protocol server for managing Apache Beam data pipelines across multiple runners including Flink, Spark, Dataflow, and Direct. It provides a standardized API for data engineers, AI/LLM developers, and DevOps teams to create, monitor, and control pipelines.

How to use Apache Beam MCP Server?

Clone the repository, create a Python virtual environment (3.9+), install dependencies from requirements.txt, then start the server with python main.py --debug --port 8888. For runner-specific configurations, set CONFIG_PATH environment variable. Pipelines are submitted via HTTP POST to /api/v1/jobs with job parameters in JSON. Use the Python client library or curl for interaction.

Key features of Apache Beam MCP Server

Multi-runner support: Flink, Spark, Dataflow, and Direct
MCP compliant for AI/LLM integration
Pipeline management: create, monitor, and control jobs
Easy to extend with new runners or custom features
Production-ready with Docker, Kubernetes, and monitoring

Use cases of Apache Beam MCP Server

Manage Apache Beam pipelines with a consistent API across different runners
Enable AI-controlled data pipelines through the MCP standard
Simplify pipeline operations and monitoring for DevOps teams
Deploy and scale pipelines in Docker or Kubernetes environments

常见问题

Which Apache Beam runners are supported?

The server supports Flink, Spark, Dataflow, and Direct runners.

How can I deploy the server in a containerized environment?

Pre-built Docker images are available on GitHub Container Registry. Kubernetes deployment is supported via `kubectl apply` or Helm chart, with detailed instructions in the deployment guide.

What monitoring and observability features are included?

Prometheus metrics are exposed at `/metrics`, pre-configured Grafana dashboards are available, structured JSON logging is used, and a health check endpoint is provided at `/health`.

What are the runtime dependencies?

The server requires Python 3.9+ and Apache Beam 2.50.0. It can run with the Direct runner (no external dependencies) or with Flink, Spark, or Dataflow runners if those are installed.

What API endpoints are available for pipeline management?

The server implements MCP standard endpoints (`/tools`, `/resources`, `/contexts`) plus pipeline-specific endpoints at `/api/v1/jobs` for submitting, monitoring, and managing jobs.

其他分类下的更多 MCP 服务器

Reactive Resume

amruthpillai

A one-of-a-kind resume builder that keeps your privacy in mind. Completely secure, customizable, portable, open-source and free forever. Try it out today!

39.5K

Glasswarp

See and control a real Windows PC you own — from any MCP client, locally or remotely. Observe (UIA + screenshots), click/type/drag/scroll, launch apps, owner Live View. BYOH: your machine, your key.

YOUB

Connect your AI assistant to your own YOUB endurance training account. Read recorded activities with full analysis, planned sessions, race goals, wellness (HRV, sleep, resting HR) and your athletic profile. Create or mov

Activepieces

activepieces

AI Agents & MCPs & AI Workflow Automation • (~400 MCP servers for AI agents) • AI Automation / AI Agent with MCPs • AI Workflows & AI Agents • MCPs for AI Agents

23.2K

FastMCP v2 🚀

jlowin

🚀 The fast, Pythonic way to build MCP servers and clients.

26.0K