How we cut agent cost 8% without touching a prompt

When we opened our own traces for the first time, the most expensive span in the system was not a model call. It was a retry wrapper that re-sent a four-thousand-token context three times on every timeout.

Finding the span

Sorting spans by cost instead of latency put the retry wrapper at the top in seconds. The same view had been hidden in logs for months, because logs report each attempt as a separate line.

The change

We moved the retry below the context builder so only the failed call repeats, and capped retries at two. Nothing about the prompts changed.

const draft = await retry(() => model.call(context), { max: 2 })

const draft = await retry(() => model.call(context), { max: 2 })

const draft = await retry(() => model.call(context), { max: 2 })

The result

Cost per run fell by 8% and p95 latency dropped by 220 ms. Evals stayed flat, which is the part we care about most.