Citations and Grounding
Grounding means the model answers only from provided sources; citations make each claim traceable to the exact passage that supports it. Together they are the trust mechanism of RAG: they let users verify, let you detect hallucination (an uncited or unsupported claim is a red flag), and are mandatory in high-stakes domains. Applied-AI interviews probe it because 'it gave a great answer' is worthless if you cannot tell whether it is true, and citations are how production AI earns trust.
TL;DR: Grounding instructs the model to answer only from the retrieved sources, and citations attach each claim to the exact passage that supports it. Together they are RAG's trust mechanism: users can verify, you can detect hallucination (an unsupported or uncited claim is a red flag), and high-stakes domains (legal, medical, finance) require it. An impressive answer is useless if no one can tell whether it is true, citations are how AI output becomes checkable, and checkable output is what production trusts.
Grounding: answer from sources, not memory
In RAG, grounding means the prompt instructs the model to base its answer on the provided context and to say "I do not know" if the context lacks the answer. This is the core lever against hallucination: the model paraphrases supplied facts rather than inventing from its weights. Grounding has a hard ceiling, retrieval recall: if the relevant passage was never retrieved, no instruction can save you, the model either abstains or makes something up. So grounding only works when retrieval actually surfaced the material and the model is told it may abstain.
Citations: make every claim checkable
Citations go further: each statement in the answer points to the specific source passage that supports it (a document, a chunk, ideally a character span). This does three things:
- Verification. Users (and reviewers) can click through and confirm, essential when wrong answers have consequences.
- Hallucination detection. A claim with no supporting source, or one the cited source does not actually back, is a red flag you can flag or block (a faithfulness check).
- Trust and compliance. Legal, medical, and financial tools require traceability; an answer you cannot trace is unusable there.
When sources disagree, the answer should surface the conflict rather than silently picking one.
How you actually wire citations
Two patterns dominate, and they trade reliability against cost:
| Approach | How it works | Failure mode | When to use |
|---|---|---|---|
| Inline markers | Number each chunk [1]..[k], tell the model to emit [2] after each sentence | Model cites a plausible-but-wrong chunk, or cites nothing | Default, cheap, one call |
| Post-hoc attribution | Generate answer, then a second pass (or an NLI model) maps each sentence back to the best-supporting span | Misses claims the source only weakly implies | High-stakes, when you can afford a second model pass |
| Structured spans | Force JSON of {claim, doc_id, start, end} via constrained decoding | Verbose, brittle on long answers | UIs that highlight the exact source text |
The non-negotiable check is entailment, not string overlap: does the cited span actually entail the claim? A lexical-overlap heuristic passes "revenue fell 4%" against a source saying "revenue rose 4%."
Worked example: catching an unsupported claim
Retrieved chunk: "Q3 net revenue was $4.2B, up 6% year over year." The model answers: "Q3 revenue was $4.2B [1], and margins improved on cost cuts [1]." Run a per-sentence NLI check (a small cross-encoder, e.g. a fine-tuned DeBERTa) against the cited chunk:
- Sentence 1 → entailment score 0.96 → keep.
- Sentence 2 → entailment 0.11 (the chunk says nothing about margins or cost cuts) → flag or strip.
That second sentence is a grounded-looking hallucination: it borrowed citation [1] to launder an invented claim. The cheap fluency check (does it read well?) passes it; the entailment check catches it. This is exactly what production faithfulness gates do, and it is the difference between a demo and a deployable legal tool.
Why interviewers probe this
"How do you know the answer is right?" is the question every AI product must answer, and citations/grounding are the mechanism. A strong response treats them as the trust layer: grounding to reduce hallucination, citations to make claims verifiable and to detect unsupported ones, and abstention when the context is insufficient. The follow-up they hold in reserve: "the model cited a source that does not support the claim, how do you catch it?" If your answer is "we check the citations exist," you fail; the right answer is an entailment check between claim and cited span.
Common misconceptions
- "A confident, fluent answer is trustworthy." Fluency is unrelated to truth; citations make it checkable.
- "Grounding alone removes hallucination." The model can still ignore or misuse context; citations plus faithfulness checks catch that.
- "A citation existing means the claim is supported." The model can cite a real chunk that does not back the claim; you need entailment, not just a marker.
- "Just cite the document." Cite the passage/span so the claim is actually checkable, not a whole 50-page PDF.
Key takeaways
- Grounding makes the model answer from retrieved sources (and abstain otherwise), the core hallucination lever; retrieval recall is its ceiling.
- Citations attach each claim to its supporting passage, enabling verification and hallucination detection.
- Verify with entailment between claim and cited span, not string overlap or mere marker presence.
- They are the trust layer of RAG and mandatory in high-stakes domains; surface source conflicts rather than silently choosing one.
Check yourself before an interviewer does. Answer from memory first.
You have citations on every claim. What check actually tells you a cited span supports its claim?