MCP.so
Sign In
Testingmanual

Flaky Test Triage

Run failing tests repeatedly, classify each failure as flaky or real, and fix only confirmed regressions.

Kickoff prompt

Start the "Flaky Test Triage" loop.

Goal: classify failing tests as flaky vs real and fix only real regressions
Max iterations: 5
Between iterations run: npm test -- --testPathPattern=<failing-suite>
Exit when: every failure is classified and real regressions are fixed or explicitly deferred

Step 1: Run the failing suite multiple times. Classify each failure, fix real ones, and document flaky behavior.

Self-pace this loop. After each iteration, run the check command, read the output, and only continue if the exit condition is not met. Stop when the exit condition passes or max iterations is reached. Give a short status update each pass.

Basic information

Category

Testing

Goal

classify failing tests as flaky vs real and fix only real regressions

Max iterations

5

Check command

npm test -- --testPathPattern=<failing-suite>

Exit when

every failure is classified and real regressions are fixed or explicitly deferred

Supported agents

Claude Code, Cursor

Steps

  1. 1

    Run failing tests

    Run the failing test file or suite 3–5 times. Record pass/fail pattern per test.

    npm test -- --testPathPattern=<failing-suite>
  2. 2

    Classify failures

    Label each failure as flaky (intermittent) or real (consistent). Note timing, ordering, or env dependencies.

  3. 3

    Fix real failures

    Fix confirmed real failures with minimal changes. For flaky tests, propose stabilization (retries, isolation, mocks).

  4. 4

    Confirm stability

    Re-run the suite multiple times to confirm real failures are gone and flakiness is reduced or documented.

    npm test -- --testPathPattern=<failing-suite>

More Testing loops

Testing

Build Until Green

Run the production build, fix compile and bundling errors, and loop until the build succeeds.

Testing

Coverage Until Threshold

Add focused tests until coverage meets your threshold (e.g. 80%), without changing production behavior unnecessarily.

Testing

Pre-Commit Guard

Hook that runs tests before git commit commands to block commits when the suite is red.

Testing

Post-Edit Test Guard

Hook-based loop that runs related tests after file edits to catch regressions early.

Testing

Format Until Clean

Run the formatter, fix any remaining style issues, and loop until the codebase formats cleanly with no diffs.

Testing

Autoloop TDD

Test-first loop: write a failing test, implement the minimum code to pass, refactor, and repeat.