Definition
In AI, hallucination refers to instances where a model generates outputs that are fluent but factually incorrect or nonsensical. It is especially common in large language models and generative AI.
Purpose
Studying hallucinations helps improve model reliability and safety. It allows developers to design safeguards to detect and reduce inaccurate outputs.
Importance
- Reduces trust in AI if unaddressed.
- Can cause harm in sensitive applications like healthcare or law.
- Highlights limitations of current generative models.
- Drives research in factual grounding and retrieval methods.
How It Works
- Model receives a prompt or query.
- Generates output based on learned patterns, not fact verification.
- May produce plausible-sounding but incorrect results.
- Detection and correction techniques are applied (e.g., RAG).
Examples (Real World)
- ChatGPT occasionally produces incorrect facts when prompted.
- Google Bard’s initial demo showed factual mistakes.
- AI-generated medical advice sometimes contains inaccuracies.
References / Further Reading
- “Reducing Hallucinations in Large Language Models” — arXiv preprint.
- NIST AI Risk Management Framework.
- Mitchell et al. “Model Cards for Model Reporting.” ACM FAccT.