Your agent got the task right once, but will it get it right again?

The scenario is familiar to anyone who has put an AI agent into real production use: in rehearsal it solves the task, but in the live demo it picks a different path and fails on the same request. On stage, it's embarrassing. In production, it's a reliability problem, and for critical work (reconciling a financial transaction, checking an obligation in a contract) it becomes a dealbreaker.

An article published by Hugging Face, authored by researchers from IBM Research, tackles exactly this point: most benchmarks hide this variability behind an average. The piece introduces the Consistency Analyzer and a new type of guideline in the open-source toolkit ALTK-Evolve, aimed at measuring and improving consistency, not just average accuracy.

The metric almost nobody reports

Standard agent evaluation uses Mean@k: run the benchmark k times and average the success rate. It's the number on every leaderboard, the famous "77% accuracy." It answers "how good is this agent on average?" but it doesn't answer the question a real user asks: if I repeat the exact same request, will it work again?

For that, there's Pass^k: the fraction of tasks in which the agent succeeds in all k independent runs. And here's the important distinction the article is careful to highlight, because the mix-up is easy to make:

• Pass@k (optimistic): at least one of the k attempts passed. It's the right question when you can verify and retry, common in code-generation papers.
• Pass^k (pessimistic): every attempt needs to pass.
• The relationship is always Pass^k ≤ Mean@k ≤ Pass@k.

The study's numbers show the size of the gap. A ReAct agent running GPT-4.1 on AppWorld (test_normal, 168 tasks) scores Mean@5 of 77.4%, genuinely strong. But Pass^5 is only 53.0%. In other words, almost a quarter of the benchmark consists of tasks the agent sometimes solves and sometimes doesn't, with nothing changing between runs. This difference (Mean@k minus Pass^k) is what the authors call the consistency gap, here 24.4 percentage points, reaching 30 points on the hard tasks.

The point that changes the mind of anyone building these systems: this isn't a lack of capability that a bigger model fixes. It's an orthogonal axis. An agent can be capable and inconsistent at the same time.

Why the agent changes its mind

Every time an LLM agent decides something (which API to call, which argument to pass, whether it's worth retrying), that decision comes out of a probability distribution over the next tokens. What matters is the shape of that distribution.

A sharp distribution concentrates almost all the mass on a single token: the runners-up trail far behind, and the same choice comes out run after run. A flat distribution spreads similar mass across several near-tied tokens, and which one wins is practically a coin flip.

The cruel detail is that this instability survives your decoding settings. The article is explicit: the experiment's agent runs at temperature 0.0, so none of this variance is ordinary sampling. Greedy decoding and a fixed seed only govern how the distribution turns into a token, not the distribution itself. On a hosted endpoint, platform effects (GPU floating-point non-associativity, request batching) nudge the numbers just enough to flip a technical tie. And since a trajectory chains together dozens of decisions, a small per-step flip chance accumulates into a high chance that some run turns out different. That's where the 24-point gap comes from.

Diagnosing before fixing

The problem turns into a search: which steps in that trajectory were the "plans," and what to do with them once identified. The answer is a two-stage pipeline that fits into the machinery ALTK-Evolve already had (it already turned the agent's past trajectories into reusable guidelines injected at inference time).

1. Detect, the Consistency Analyzer. Given a single recorded trajectory, the analyzer reprocesses each decision step with controlled resampling, measuring how much the model's output actually varies there. In practice it's one extra call to the model per decision step, done once, offline, requesting k completions at once (k=5 by default), replayed against the already-recorded context. There's no new tool call, no new interaction with the environment, and no second end-to-end rollout of the task. Each step gets a consistency score on a scorecard that points exactly to which decisions are at risk of flipping in the next run.

Two practical points here matter more than the rest for anyone operating in production: detection is fully black-box (no logits, no model internals, just the trace you already have) and doesn't need ground truth or live replay. That's what makes the technique usable on real traffic, where you often can't even re-run a task once.

2. Generate targeted guidelines. Each flagged step becomes a candidate guideline in ALTK-Evolve's standard format. The article shows a real example generated by GPT-4.1 from the task "how many activities from my bucket list are in my SimpleNote note?":

• When counting checkbox-style markers in a note's content, use line-anchored regex instead of simple substring counting, because titles often repeat the marker symbol in a caption line.
• Always verify note search results by checking for multiple matches and confirming the correct note before proceeding.

Notice: none of this is task-specific trivia. String-counting bugs and unverified search results are high-uncertainty decision points that show up across many tasks. The analyzer targets instability, not failure, meaning it catches steps the agent got right this time but could get wrong next time.

The numbers: cutting the gap without losing accuracy

Generating consistency guidelines from a single baseline trajectory per task and testing on 5 new runs, the aggregate result was:

• Pass^5 rises from 53.0% to 69.0%, while Mean@5 rises from 77.4% to 81.0%. The gap drops from 24.4pp to 12.0pp, roughly half.
• Nearly a third of previously inconsistent tasks now succeed in every run.
• The biggest gains are in the middle and top of the difficulty range: Medium +22.9pp (+44% relative) and Hard +14.3pp (+45% relative). Easy rises +12.2pp, with less room to improve.
• Mean@5 never drops. Preserving average accuracy was a requirement, not a bonus: a system that raises Pass^5 by trading off Mean@5 would just be pushing the reliability problem to another corner.

There's evidence the guidelines generalize rather than just patching a single trajectory. Applied to a different but related task in the same AppWorld scenario, they still raise Pass^5 by +13.0pp, only 3 points below the same-task number. And on a weaker model, gpt-oss-120b, the gain on a similar task (+8.7pp) actually exceeded the gain on the same task (+6.0pp), suggesting they're capturing reusable failure patterns rather than memorizing a trajectory.

What changes for those shipping agents

The article's direct lesson for anyone who already has an agent in production or about to ship one:

• Report Pass^k alongside Mean@k. Averages don't distinguish a reliable agent from a lucky one. Even at k=3, a gap you didn't know you had already shows up.
• Expect the gap to grow with difficulty. It's exactly in your hardest tier that a single average number is most misleading.
• Don't reach for a bigger model first. Consistency is orthogonal to capability. A stronger model raises Mean@k but doesn't necessarily close the consistency gap.
• Diagnosis doesn't need a grader or replay. One extra LLM call per decision step is enough.

The Consistency Analyzer and guideline generation are already in the ALTK-Evolve open-source repository, and the full methodology is in the technical report on arXiv. For the Brazilian developer who today measures their agent by "it passed in the demo," the practical takeaway comes before the tool itself: start running the same request multiple times and look at how many it passed in all of, not on average. That's the number the user actually feels.