Verifiers First, Rubrics Second

Published 2026-10-11

Evaluating LLM Agents, part 4. Series page and full outline.

Every grader is a program, a model, or a person. Programs are the cheapest and the only ones that give the same answer twice; people are the most expensive and, when it comes down to it, the most accurate, which is why we use them to calibrate the models in between (Anthropic, 2026). Which of the three a task gets comes down to one question: can a program tell whether the task was done?

Figure 1. The decision depends on the task rather than the technology: verifiable tasks get verifiers, decomposable judgments get rubrics, and holistic judgments get pairwise preference.

Verifiable tasks

A task is verifiable when success is something a program can compute from the end state. The tests pass, or they don't; the answer matches; the row exists; the file compiles; the simulated customer's goal state matches what's in the database (Yao et al., 2024).

What makes verifiers so valuable is that they're deterministic, cheap, and happy to run millions of times, and that last property is why they can serve as training rewards and not just as evals. RLVR is, at bottom, just using the verifier as the reward (Lambert et al., 2024; DeepSeek-AI, 2025), and most of the agent benchmarks people take seriously are graded by verifiers too: SWE-bench by tests (Jimenez et al., 2024), WebArena by state (Zhou et al., 2023), OSWorld by scripts that go and inspect files, configuration, and interface elements (Xie et al., 2024).

If a verifier exists for your task, use it, and save the judge budget for the things it can't see.

Non-verifiable tasks

There's no program that decides whether a research memo is good, or whether a support reply has the right tone, or whether a design document or a piece of medical advice holds up. A rubric is how you give a task like that some of the structure of a verifiable one: you break the judgment into criteria that are specific enough for a model to check one at a time.

The recent examples are more convincing than I expected them to be. HealthBench grades 5,000 health conversations against 48,562 criteria written by 262 physicians (Arora et al., 2025). PaperBench breaks the replication of a research paper into a tree of 8,316 gradable leaves, co-written with the papers' own authors, and ships a separate benchmark for the judge itself (Starace et al., 2025). Used as training signals, rubric rewards beat Likert-scale judge rewards by up to 31% relative on HealthBench (Gunjal et al., 2025); instruction-specific checklists were the only method that improved every benchmark tested (Viswanathan et al., 2025); and a system of more than 10,000 rubrics extended RLVR to open-ended writing (Huang et al., 2025). The common thread, as far as I can tell, is that the criteria are specific to the instance, binary or nearly so, and written by people who could have done the task themselves.

When a rubric is the wrong tool

There are four situations where I'd push back on reaching for a rubric.

The first is when a verifier exists. Asking a judge whether the tests pass, instead of running the tests, adds noise, cost, and a surface for the agent to game, and gains you nothing. If code quality matters too, make correctness the gate and the rubric a secondary score, which is roughly how Anthropic describes doing coding evals in practice (Anthropic, 2026).

The second is when the judgment is holistic. "Is this essay good?" doesn't decompose, and a checklist ends up rewarding writing that is shaped like a checklist. Pairwise comparison, the MT-Bench and Chatbot Arena approach (Zheng et al., 2023; Chiang et al., 2024), works better, provided you control for the known biases: changing nothing but the order in which two candidates appeared was enough to make Vicuna-13B beat ChatGPT on 66 of 80 queries (Wang et al., 2023); controlling for length raised AlpacaEval's correlation with human rankings from 0.94 to 0.98 (Dubois et al., 2024); and models favor their own outputs, more so the better they are at recognizing them (Panickssery et al., 2024).

The third is when nobody has looked at any outputs yet. Shankar et al. (2024) found that people need criteria to grade outputs, but they also need to grade outputs before they can say what their criteria are. A rubric written before you've read a stack of real trajectories is a list of guesses. It should come out of error analysis on real runs, and it should get revised whenever the system changes.

The fourth is when the rubric is the only signal in a long optimization. A rubric scored by a model is a reward model with some extra structure, and reward models get over-optimized (Gao et al., 2022). The protection is a held-out set the optimizer never sees, graded by people on some regular cadence.

Verifiers have error rates too

A verifier is only as good as its coverage, and in my experience coverage is almost always worse than it looks. When people manually screened SWE-bench, they found that 31.08% of the passing patches were suspicious because the tests were too weak to catch a wrong fix, and that 32.67% had the solution leaked in the issue text; filtering out both dropped a top agent from 12.47% to 3.97% (Aleithan et al., 2024). Zhu et al. (2025) surveyed agentic benchmarks and found reward-design errors that mis-estimate performance by up to 100% in relative terms, including one benchmark that counted an empty response as a success. The error runs the other way as well: rule-based checks underreport web-agent success because they miss perfectly valid routes to the goal (Lù et al., 2025).

So verifier quality is itself something to measure, as false-pass and false-fail rates on a labeled sample. And, as Part 3 noted, agents will exploit whatever gaps the verifier has if they can reach it (Von Arx et al., 2025).

Judge hygiene

When a model is doing the grading, a few habits keep the number usable.

• Use binary criteria rather than Likert scales. Scales are hard to apply consistently, and rubric criteria turned out to align smaller judges better and reduce variance across judge sizes (Gunjal et al., 2025; Husain, 2024).
• Calibrate against people on a labeled slice and report the agreement, then do it again whenever the system changes. Online-Mind2Web's judge gets to about 85% agreement with human judgment (Xue et al., 2025), and PaperBench built a benchmark for its judge before it trusted the judge (Starace et al., 2025).
• For pairwise judgments, swap the positions and control for length (Wang et al., 2023; Dubois et al., 2024).
• Give the judge the evidence, meaning the end state or the artifact itself, not just what the transcript claims about them.
• If you need a judge you can inspect, open ones exist (Kim et al., 2024), and Gu et al. (2024) survey the design space.

A worked example

Here is one task from a support workflow, graded with a verifier, a rubric, and a calibrated judge, with made-up but realistic numbers.

The task. A customer writes in to say order 4411 arrived damaged and they want a refund. The policy says: full refund if it's reported within 14 days and photos are attached, store credit otherwise, and anything over 200 USD has to go to a manager. The starting state is order 4411, 86 USD, delivered nine days ago, photos attached. The agent has five tools: look up an order, issue a refund, issue credit, escalate, and reply.

The verifier. Done here is a property of the database, so a program can check it, and both lists get written down before anything runs.

required:
  refunds:        one row, order 4411, amount 86.00, status issued
  tickets[4411]:  status resolved
  messages_sent:  at least one reply to the customer
forbidden:
  credits:        none issued
  escalations:    none (the amount is under the manager threshold)
  refunds:        none on any other order, and never more than one

Say we run 500 trials and 400 of them pass. That's the correctness number, 80%, and no model was involved in producing it.

The rubric. Reply quality is the part that can't be verified, so it gets judged, and only on the 400 trials that already passed the verifier, since correctness is the gate. The support lead read thirty real replies before writing four binary criteria, because (as usual) the criteria they expected to need weren't the ones the replies were actually failing on:

1. States the amount refunded and that it has been issued.
2. Says when to expect it (five to seven business days, per policy).
3. Promises nothing outside policy.
4. Doesn't ask for information that's already in the ticket, like the photos.

A model grades each criterion pass or fail, with the reply, the ticket, and the policy in front of it, and a reply passes only if all four do. The judge comes from a different model family than the agent, which sidesteps self-preference (Panickssery et al., 2024). It reports 300 of 400 passing, or 75%.

The calibration. The support lead then labels a random 50 of the 400 replies by hand and gets 38 passes and 12 fails. The judge agrees with them on 45 of the 50, which is 90% agreement and sounds reassuring until you look at where the disagreements are. The judge passes 37 of the 38 human passes (a true positive rate of 0.97) but it also passes 4 of the 12 human fails (a true negative rate of 0.67). In other words it's lenient on the replies that matter most.

Figure 2. Left, the 50 hand-labeled replies split by human and judge verdicts. Right, the judge's reported pass rate and the rate after correcting for its error rates.

Those two rates are enough to correct the judge's 75%. If $p$ is the true pass rate, then what the judge observes is $q = \mathrm{TPR}\cdot p + (1 - \mathrm{TNR})(1 - p)$, and solving for $p$ gives

$$p = \frac{q - (1 - \mathrm{TNR})}{\mathrm{TPR} - (1 - \mathrm{TNR})} = \frac{0.75 - 0.33}{0.97 - 0.33} \approx 0.65.$$

That's the Rogan-Gladen estimator from epidemiology, where the identical problem comes up with imperfect screening tests (Rogan and Gladen, 1978). The corrected reply-quality number is about 65%, not 75%, so the result for the whole workflow reads: 80% of refunds were processed correctly, and about 65% of those came with a reply the support lead would accept.

Fifty labels don't get you much more than a point estimate. The true negative rate here rests on twelve examples, so its interval is wide and the corrected rate inherits all of that uncertainty (Miller, 2024). What fifty labels did establish is which way the judge is biased, and that the next labels should go into the failing class. Part 7 gets into how many labels are enough and how to size the intervals properly.

Tenets

• Choose the grader from the task: a verifier if a program can decide, a rubric if experts can agree on a checklist, pairwise preference otherwise.
• Do not rubric-grade what a verifier can check.
• Derive rubrics from real outputs rather than writing them in advance.
• Measure the verifier: false passes, false fails, and whether the agent can reach it.
• Calibrate every judge against people and keep a human-graded holdout.
• A judge's agreement rate is not its error rate; measure true positive and true negative rates and correct the pass rate with them.

References

Anthropic (2026). Demystifying evals for AI agents. Anthropic Engineering.
Yao, S. et al. (2024). τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains.
Lambert, N. et al. (2024). Tülu 3: Pushing Frontiers in Open Language Model Post-Training.
DeepSeek-AI (2025). DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning.
Jimenez, C. E. et al. (2024). SWE-bench: Can Language Models Resolve Real-World GitHub Issues? ICLR.
Zhou, S. et al. (2023). WebArena: A Realistic Web Environment for Building Autonomous Agents. ICLR 2024.
Xie, T. et al. (2024). OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments. NeurIPS.
Arora, R. K. et al. (2025). HealthBench: Evaluating Large Language Models Towards Improved Human Health.
Starace, G. et al. (2025). PaperBench: Evaluating AI's Ability to Replicate AI Research.
Gunjal, A. et al. (2025). Rubrics as Rewards: Reinforcement Learning Beyond Verifiable Domains.
Viswanathan, V. et al. (2025). Checklists Are Better Than Reward Models For Aligning Language Models.
Huang, Z. et al. (2025). Reinforcement Learning with Rubric Anchors.
Zheng, L. et al. (2023). Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena. NeurIPS.
Chiang, W.-L. et al. (2024). Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference. ICML.
Wang, P. et al. (2023). Large Language Models are not Fair Evaluators. ACL 2024.
Dubois, Y. et al. (2024). Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluators. COLM.
Panickssery, A., Bowman, S. R., and Feng, S. (2024). LLM Evaluators Recognize and Favor Their Own Generations. NeurIPS.
Shankar, S. et al. (2024). Who Validates the Validators? Aligning LLM-Assisted Evaluation of LLM Outputs with Human Preferences. UIST.
Gao, L., Schulman, J., and Hilton, J. (2022). Scaling Laws for Reward Model Overoptimization. ICML 2023.
Aleithan, R. et al. (2024). SWE-Bench+: Enhanced Coding Benchmark for LLMs.
Zhu, Y. et al. (2025). Establishing Best Practices for Building Rigorous Agentic Benchmarks.
Lù, X. H. et al. (2025). AgentRewardBench: Evaluating Automatic Evaluations of Web Agent Trajectories.
Von Arx, S., Chan, L., and Barnes, B. (2025). Recent Frontier Models Are Reward Hacking. METR.
Husain, H. (2024). Your AI Product Needs Evals.
Xue, T. et al. (2025). An Illusion of Progress? Assessing the Current State of Web Agents.
Kim, S. et al. (2024). Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Models. EMNLP.
Gu, J. et al. (2024). A Survey on LLM-as-a-Judge.
Rogan, W. J. and Gladen, B. (1978). Estimating prevalence from the results of a screening test. American Journal of Epidemiology, 107(1), 71-76.
Miller, E. (2024). Adding Error Bars to Evals: A Statistical Approach to Language Model Evaluations.