· June 19, 2026 · 3 min read
Evaluating Models on Your Own Data
Public benchmarks tell you how a model performs on public benchmarks. Here's the evaluation harness we actually trust before a deployment ships.
A model's position on a public leaderboard has almost no predictive power over how it will perform on your corpus, your query patterns, and your definition of a correct answer. This isn't a controversial claim inside the field — it's well understood — but it's still routinely ignored at the point where a model actually gets selected for a deployment, because building a real evaluation set is slower and less satisfying than reading a leaderboard.
Why the gap exists
Public benchmarks are built to be reusable across every model, every lab, every paper. That reusability requires them to be generic: broad-domain knowledge, synthetic reasoning puzzles, code that resembles nothing in a maintenance manual or a matter archive. A model can genuinely excel at that and still fail at the specific thing your deployment needs — extracting a clause type that only appears in your contract templates, answering in the register your users actually expect, correctly declining to answer when your corpus doesn't cover the question at all.
The last one matters more than it sounds. Public benchmarks rarely reward appropriate refusal the way a production deployment needs to. A model that confidently answers from outside its retrieved context looks fine on a leaderboard and is a liability in a knowledge base.
What we build instead, before anything ships
A task-specific gold set. Real or representative queries against the actual corpus, with answers reviewed by someone who knows the domain — not generated by another model and rubber-stamped. This is the slow part, and it's the part that can't be skipped without the rest of the evaluation being theatre.
Explicit refusal cases. Queries the corpus genuinely can't answer, included on purpose. A model that answers everything confidently fails these by definition, and this is often the single most informative slice of the eval set for deciding whether a system is safe to put in front of end users.
Retrieval and generation scored separately. A wrong answer can come from bad retrieval (the right chunk was never in context) or bad generation (the right chunk was there and the model ignored it). Conflating the two into one end-to-end accuracy number tells you that something failed without telling you what to fix.
Held-out drift checks. Corpora change. A gold set built in month one and never revisited stops measuring anything real by month six. We re-sample and re-review on a schedule tied to how fast the underlying corpus actually changes, not a fixed calendar interval.
A model that's 4% behind the field leader on a public benchmark and 15% ahead on your gold set is the better model for you. The leaderboard doesn't know your corpus exists.
The honest limitation
Building a real gold set requires domain expert time, and domain experts are usually the busiest people in the building. We've had projects stall for weeks waiting on review capacity that had nothing to do with the technical work. There's no shortcut here worth taking — a fast evaluation built on a rubber-stamped gold set is worse than no evaluation, because it produces false confidence instead of an honest "we don't know yet."