Shipping v2.4: evals as a deploy gate
A version now ships only when its evals clear the bar you set. Here is how it works.
T
Maya Brandt
4 min
Starting today every deploy runs your eval suite first and compares the result with the version in production.
Set the bar once
Choose a pass rate, a latency budget and a cost ceiling. Gates apply to every project that inherits the policy.
What happens on a fail
The deploy stops, the failing cases are linked in the alert, and the previous version keeps serving traffic. Nobody has to remember to roll back.
