Sail MCP Server for Spark SQL

Name: Sail MCP Server for Spark SQL
Author: lakehq

@lakehq

Sail MCP Server for Spark SQL について

Drop-in Apache Spark replacement written in Rust, unifying batch processing, stream processing, and compute-intensive AI workloads.

#sail #spark #sql #data-analytics

基本情報

カテゴリ

データベース

ライセンス

Apache-2.0

ランタイム

rust

トランスポート

stdio

公開者

lakehq

投稿者

Heran Lin

設定

以下の設定を使って、このサーバーを MCP 対応クライアントに追加してください。

{
  "mcpServers": {
    "sail": {
      "command": "sail",
      "args": [
        "spark",
        "mcp-server",
        "--transport",
        "stdio"
      ]
    }
  }
}

ツール

ツールは検出されませんでした

ツールは README から自動的に抽出されます。メンテナーは ## Tools という見出しの下に記載することで、このタブに反映できます。

概要

What is Sail MCP Server for Spark SQL?

Sail MCP Server for Spark SQL is a drop-in Apache Spark replacement written in Rust, providing a Spark SQL and DataFrame API server over the Spark Connect protocol. It unifies batch processing, stream processing, and compute-intensive AI workloads on a distributed engine. Based on derived TPC-H benchmarks, it is ~4× faster and 94% cheaper than Spark.

How to use Sail MCP Server for Spark SQL?

Install pysail and pyspark-client, then start the server via the command sail spark server --port 50051 or using the Python API with SparkConnectServer(port=50051).start(). Connect a PySpark client session by setting the remote address to sc://localhost:50051.

Key features of Sail MCP Server for Spark SQL

Drop-in compatibility with Spark SQL and DataFrame API.
100% Rust-native engine with no JVM overhead.
Supports Delta Lake and Apache Iceberg table formats.
Integrates with AWS Glue, Unity Catalog, Hive Metastore, and OneLake.
Lightning-fast Python UDFs with zero-copy Arrow data sharing.
Lightweight, stateless workers for elastic scaling.

Use cases of Sail MCP Server for Spark SQL

Migrating existing PySpark workloads to a faster, cheaper engine.
Running batch ETL and data analytics pipelines.
Real-time stream processing using Spark-compatible APIs.
Compute-intensive AI workloads requiring high performance.

FAQ from Sail MCP Server for Spark SQL

How does Sail compare to Apache Spark?

Sail is a drop-in replacement that is ~4× faster and requires 94% less hardware cost, with zero shuffle spill per derived TPC-H benchmarks.

What are the runtime dependencies?

Python and the PySpark client package (pyspark-client). The server itself is a standalone Rust binary with no JVM required.

Where does data live?

Data can be read from and written to AWS S3, Azure, Google Cloud Storage, HDFS, Cloudflare R2, HTTP/HTTPS, Hugging Face, and local filesystems.

Is my existing Spark code compatible?

Yes, existing PySpark code works out of the box when the client connects over Spark Connect. An experimental compatibility check script scans your codebase for supported functions.