Blog
What agent traces teach us.
Notes from measuring AI coding agents on public runs: why they fail, which alerts hold up, and what we changed in Earlyman because of it.
Most failed coding-agent runs end quietly
Most failed public coding-agent runs leave no failing step behind. The most useful fact in a trace is whether the last change was ever checked.
Read the post →An alert that fires on 91 % of good runs is noise
A signal is only useful if it fires on failed runs more often than on good ones. Measure that gap before you save it, not after your team stops reading alerts.
Read the post →Why we kept rules over a model that made things up
A small local model wrote fluent, specific explanations of failed agent runs. On 29 of 40 runs they contained facts the run did not support. A fixed, explainable cause pipeline won.
Read the post →