Lab Note · April 18, 2026 · 3 min read
Retrieval That Cites Its Sources
Why every answer our systems produce carries a pointer back to the exact paragraph it came from — and what that costs at query time.
A retrieval system that can't tell you where an answer came from isn't actually saving anyone time — it's just moving the verification work from "read the document" to "hope the model didn't hallucinate the document." For anything wired into a workflow with real consequences (a contract clause, a maintenance procedure, a compliance answer), that trade is a bad one. So every system we ship treats citation as a first-class output, not a nice-to-have wrapped around the answer afterward.
The naive version doesn't work
The obvious approach is to ask the model to cite its sources in the same generation pass that produces the answer. It's cheap, and it's wrong more often than it looks. Language models are fluent enough to produce a plausible-looking citation — a page number, a section heading — that has nothing to do with which chunks were actually in the context window. The failure is quiet: the answer might even be correct, and the citation still fabricated.
We stopped trusting model-generated citations entirely. Citation is a retrieval-time fact, not a generation-time claim.
What we do instead
The pipeline separates "what did we retrieve" from "what did the model say" and reconciles them after generation, not before:
- Chunk-level provenance. Every chunk carries its source document, page or section, and byte offset from ingestion onward. This travels with the chunk through re-ranking — it's metadata, not something we reconstruct later by fuzzy-matching text.
- Constrained context assembly. The generation prompt only ever contains chunks we can point back to. No summarized-then-discarded intermediate step where provenance could get lost.
- Post-hoc grounding check. After generation, we run a lightweight entailment pass: does each sentence in the answer follow from the specific chunks cited alongside it? Sentences that don't pass get flagged rather than silently shipped — in practice this catches the cases where the model drifted from the retrieved material into something that sounds right.
The citation shown to the user is always the chunk actually used, resolved back to the exact paragraph in the original document — not a page-level "see document X" gesture.
What it costs
Grounding checks add latency — roughly 15-20% on top of generation time in our current harness, depending on answer length. For most of our deployments that's a fine trade against a wrong answer with a false citation, but it's not free, and we've had exactly one project where the latency budget was tight enough that we shipped without the grounding pass and leaned harder on retrieval precision instead. That's the right call sometimes. It should be a deliberate one, not a default.
The moment a citation is wrong once, a user stops trusting all of them — including the correct ones. Grounding isn't about the average case; it's about not losing the whole system to the worst case.
Where this breaks down
Citation traceability gets harder, not easier, once a corpus has revision history — contracts get amended, wikis get edited, drawings get superseded. "Cite the source" implicitly means "cite the source as of when," and most of our ingestion pipelines weren't built with that question in mind from day one. It's the thing we're actively working on next, and probably worth its own Lab Note once we have something worth writing down.