p95 is a lie: tail latency in agent loops
Loops multiply latency. A look at why the slowest one percent of runs matter more than the average.
T
Ines Carvalho
8 min
In a single model call, p95 describes the unlucky one in twenty. In an agent that loops five times, the unlucky one in twenty happens on nearly a quarter of runs.
Measure the run, not the call
Report latency at the level your user feels it: the whole run. Break down by span only to find the cause.
Cap the loop
A maximum step count and a time budget turn the long tail into a controlled failure.
