Verify Action

What is Verify Action?

Verify Action is a post-action verification service for AI agent tool calls. Submit a (claim, evidence) pair and get back a 4-value verdict and a tamper-evident hosted receipt that downstream agents, CI steps, or deployment pipelines can use to continue, block, or escalate. It checks whether supplied evidence is internally consistent with the supplied claim — it does not independently access the caller’s database, code repository, or APIs.

How to use Verify Action?

Configure it as an MCP server (e.g., in claude_desktop_config.json with transport http pointing to https://verify.armadalab.dev/mcp). The agent then has a verify_action tool available. You can also call the REST API directly (POST /verify) or self-host the pure Python stdlib service (no pip install; clone and run ./start.sh).

Key features of Verify Action

• Post-action evidence verification for AI agent tool calls
• Four canonical verdicts: verified, contradicted, insufficient_evidence, unsafe_to_verify
• HMAC-SHA256-attested receipts for tamper-evident logging
• Primary verifier for code diffs; experimental verifiers for DB ops, file ops, and API calls
• Self-hostable with pure Python stdlib; no dependencies
• Built-in PII guard rejects email, phone, credit-card, and JP personal‑ID patterns

Use cases of Verify Action

• Verify a code diff matches the agent’s claimed change before merging
• Block a deployment step when evidence contradicts the claimed database operation
• Escalate to a human when insufficient evidence is provided for a claimed API call
• Ensure a downstream CI job only proceeds on a “verified” verdict from prior agent action

FAQ from Verify Action

What exactly does Verify Action examine?

It examines the supplied evidence object against the supplied claim using rule‑based verifiers. It does not access external systems; the trust boundary depends on the evidence kind — strongest for code_diff (the diff is the evidence), weaker for kinds that rely on the caller faithfully reporting external state.

What are the four possible verdicts?

verified (proceed), contradicted (block), insufficient_evidence (request more evidence), and unsafe_to_verify (escalate to human — the verifier itself raised an exception).

How does the receipt trust model work in v0?

Receipts are HMAC‑SHA256 symmetric‑key signatures — they prove this service issued the verdict under a known key id. They are tamper‑evident hosted log entries, not third‑party cryptographic attestations. Public‑key (Ed25519) receipts are planned for a future v1.

What are the current limitations?

The service is rule‑based only (no LLM‑as‑judge), does not decompose sub‑claims, and does not correlate traces across calls. The generic evidence handler is conservative and often returns insufficient_evidence.

How does Verify Action handle privacy?

IP addresses are SHA‑256‑hashed with a rotating salt. Claims and evidence are written to private trace logs with 30‑day retention. A structural PII guard rejects email, phone, JP postal codes, national‑ID‑shapes, passport patterns, and credit‑card‑shape digits. Individual traces stay private; only aggregate findings may be published.