Utah Salt Lab MCP Server

@Utah-SaLT-Lab

リポジトリ

基本情報

カテゴリ

その他

トランスポート

stdio

作者

Utah-SaLT-Lab

投稿者

Shih-Chieh Dai

Utah Salt Lab MCP Server について

概要はまだありません

#utah-salt-lab#code-evaluation#security

Utah Salt Lab MCP Server の設定

標準の設定はありません

このサーバーの README には解析可能な MCP 設定ブロックが含まれていません。インストール手順はリポジトリをご確認ください。

リポジトリ

Utah Salt Lab MCP Server のツール

ツール一覧はまだありません

この掲載には表示できるツール一覧がまだありません。サーバーにツールがないという意味ではありません。対応機能はプロジェクトのドキュメントをご確認ください。

ツールは README から自動的に抽出されます。メンテナーは ## Tools という見出しの下に記載することで、このタブに反映できます。

概要

What is Utah Salt Lab?

Utah Salt Lab is a unit testing framework for evaluating code generated on the SecCodePLT benchmark. It provides containerized execution and preprocessing scripts to run unit tests on generated code and output evaluation results. It is designed for researchers studying LLM secure code generation.

How to use Utah Salt Lab?

Install dependencies via pip install -r requirements.txt, build a container using Docker or CharlieCloud, then run python preprocess.py to distribute generated code into test folders, execute sh run.sh, and finally run python get_result.py to collect evaluation results.

Key features of Utah Salt Lab

Supports 1,201 SecCodePLT tasks with generated unit tests.
Containerized environment recommended via Docker or Charliecloud.
Preprocessing script distributes generated code into unit test folders.
Results aggregation script outputs final evaluation metrics.
Task IDs listed in utils/SecPLT_func_name.json.

Use cases of Utah Salt Lab

Evaluate secure code generation from large language models.
Compare code quality across different LLMs on security tasks.
Reproduce experiments from the associated arXiv paper (2503.15554).

よくある質問

What is the connection between Utah Salt Lab and SecCodePLT?

Utah Salt Lab provides unit tests for the SecCodePLT benchmark. After filtering, 1,201 of the original 1,345 tasks have unit tests.

How do I set up the container environment?

Use the provided Dockerfile or CharlieCloud. Set `CH_IMAGE_STORAGE` to a directory with sufficient space, then build with `ch-image build -t test-runner -f Dockerfile .`.

Where are the evaluation results stored?

Results are saved via `get_result.py` to the path specified with `--output`. Intermediate files are stored in `data/` subdirectories.

How should I format generated code for testing?

Store code in a `.jsonl` file with each line containing `{"task_id": "<id>", "solution": "<code>"}`. The task IDs must be among the supported 1,201.

Why is containerized execution recommended?

The generated code may contain dangerous operations such as `rm -rf`. A container isolates the test environment and prevents unintended system damage.

「その他」の他のコンテンツ

Senso MCP for Joomla

sensomedia

Senso MCP turns a Joomla 5 or 6 site into a remote MCP server. It runs inside Joomla itself (no extra server, no Node.js bridge) and is secured with OAuth 2.1 and Joomla's own ACL. The free pack lets an AI assistant crea

Reactive Resume

amruthpillai

A one-of-a-kind resume builder that keeps your privacy in mind. Completely secure, customizable, portable, open-source and free forever. Try it out today!

39.5K