AI hallucination is when a generative AI model produces output that sounds plausible and confident but is false, invented or not supported by its sources. Examples include made-up facts, fake quotes, non-existent research papers and wrong numbers. Hallucinations are one of the most important limitations of generative AI, especially in legal, medical, financial and journalistic work.
Key takeaways
- Hallucinations happen because language models generate likely-sounding text, not verified facts.
- They often look fluent and confident, which makes them hard to spot.
- Grounding answers in trusted sources, such as with retrieval, reduces them significantly.
- They can be reduced but not completely eliminated with current technology.
- Human verification is essential for anything important.
AI hallucination explained in simple terms
Imagine a very well-read person who never says “I don’t know”. Ask about a book they half-remember and they will confidently fill the gaps with details that sound right. Some will be accurate; some invented. A language model behaves similarly. It predicts the most plausible next words based on patterns, and plausible is not the same as true.
Why do AI models hallucinate?
- Next-token prediction: a decoder model is trained to produce likely text, not to check facts.
- Gaps in training data: rare, recent or niche facts may be missing, so the model fills gaps.
- Knowledge cut-off: models do not know about events after their training data ends unless given current information.
- Pressure to answer: training and evaluation have often rewarded confident answers more than admitting uncertainty.
- Ambiguous prompts: unclear questions invite guesses.
- Long outputs: small errors early can compound.
Types of hallucination
| Type | Example |
|---|---|
| Fabricated facts | Wrong founding year for a company |
| Invented citations | A realistic-looking paper or court case that does not exist |
| Fake quotes | Words attributed to a real person who never said them |
| Wrong numbers | Incorrect statistics, prices or calculations |
| Source contradiction | A summary that says something the document does not |
| Visual hallucination | A multimodal model “seeing” details not in an image |
Real-world examples
- Mata v. Avianca (2023): lawyers in New York submitted a court filing citing cases that a chatbot had invented. The judge sanctioned them.
- Moffatt v. Air Canada (2024): a Canadian tribunal held Air Canada responsible after its website chatbot gave a customer incorrect information about bereavement fares.
Both show that organisations remain accountable for AI output they rely on or publish.
How to reduce hallucinations
| Technique | How it helps |
|---|---|
| Retrieval-augmented generation (RAG) | Supplies relevant, trusted documents so answers are grounded in real sources |
| Ask for citations | Require the model to cite specific passages, then check them |
| Allow “I don’t know” | Instruct the model to say when information is missing |
| Clear, specific prompts | Reduce guessing through prompt engineering |
| Lower temperature | Reduces randomness for factual tasks |
| Structured output | Constrains answers to defined fields |
| Tools and calculators | Let the model look up data or compute instead of guessing |
| Human review | Check facts, numbers and quotes before use |
A worked example: grounding a support bot
A company’s chatbot is asked, “What is your refund period?” Without grounding, it might say “30 days”, a common answer on the internet. With RAG, the system first retrieves the company’s actual refund policy, which says 14 days, and the prompt instructs the model to answer only from that document and say “I’m not sure” if the answer is not there. The bot now answers “14 days” and links to the policy page.
Is hallucination always bad?
In creative work, such as brainstorming, fiction or design ideas, generating things that do not exist is the point. The problem arises when invented content is presented or used as fact. The right level of control depends on the task.
Limitations of current fixes
Even with retrieval, models can misread or misquote sources. Citation checks catch many errors but take time. Detection tools are imperfect. For high-stakes use, the safest approach combines grounding, constrained outputs and human verification.