Grounded generation is when a model answers from supplied source material rather than its own memory, ideally citing which passages support each claim. It is the generation half of RAG: retrieval finds the evidence, grounding ensures the answer stays faithful to it. Good grounding turns an unverifiable answer into one a user can check.
Key takeaways
- Grounding is about faithfulness: every claim traceable to the provided context.
- It is enforced through prompting, citation requirements and verification, not by retrieval alone.
- A grounded system must be able to say “this is not in the sources”.
- Faithfulness and correctness differ: a faithful answer to a wrong source is still wrong.
- It greatly reduces, but does not eliminate, hallucination.
Grounded generation explained in simple terms
A journalist writing from memory may get details wrong. A journalist writing with the documents open, quoting and footnoting each claim, can be checked. Grounding is asking the model to write like the second journalist, and refusing to accept claims it cannot point to.
How grounding is enforced
- Instruct explicitly: answer only using the provided context; if the answer is not there, say so.
- Label the context: give each passage an identifier so it can be cited.
- Require citations: ask for source markers after each claim.
- Constrain scope: forbid general knowledge unless the prompt allows it.
- Verify: optionally check each claim against the cited passage with a second pass.
- Display sources: show links so users can confirm.
A worked example
A grounding prompt might read:
“Answer the question using only the passages below. Cite the passage number after each sentence, like [2]. If the passages do not contain the answer, reply: ‘I could not find this in the available documents.’ Do not use outside knowledge.
[1] … [2] … [3] …”
Asked “What is the notice period for resignation?”, the model answers “The notice period is 60 days for permanent employees [2].” If the passages only cover probation, a well-grounded model says it could not find the answer rather than guessing 30 days from general knowledge.
Measuring grounding
| Metric | Question it answers |
|---|---|
| Faithfulness | Is every claim supported by the retrieved context? |
| Citation accuracy | Do the cited passages actually contain the claim? |
| Answer relevance | Does the answer address the question? |
| Refusal correctness | Does it decline when the answer genuinely is missing? |
| Context precision | Was the retrieved context actually needed? |
Automated evaluation often uses a second model as a judge, scoring each claim against the cited passage. Sample human review is still recommended.
Why models still drift
- The context does not contain the answer, and the model fills the gap.
- Conflicting passages leave the model to choose.
- The question invites general knowledge, such as definitions.
- Very long context buries the relevant passage.
- The system prompt is not firm enough about scope.
Practical measures
- Show citations in the interface, not just in the prompt.
- Log answers with weak or missing citations for review.
- Prefer fewer, better passages via a reranker.
- Handle conflicts explicitly: instruct the model to prefer the most recent version and note discrepancies.
- Make “not found” a good outcome, not a failure, in your evaluation.
Limitations
- Faithful does not mean true, if the source is wrong or outdated.
- Citation checking adds latency and cost.
- Over-strict grounding can make assistants unhelpfully narrow.
- Long or noisy context weakens grounding.