Run evals in CI

Fail a pull request when the pass rate drops.

LAST UPDATED

Oct 6, 2026

Run the suite on every pull request and compare it with main.

- run: npx tracewell eval run --compare main --fail-under 0.95

- run: npx tracewell eval run --compare main --fail-under 0.95

- run: npx tracewell eval run --compare main --fail-under 0.95

Reading the result

The command prints the pass rate, the change against main and the cases that moved. A non-zero exit blocks the merge.

← PREVIOUS