Hallucinations

What Are Hallucinations?

Hallucinations are outputs that read as accurate but are factually wrong. The defining feature is confidence — the model does not hedge or express uncertainty. It asserts something false as if it were undeniable fact.

Examples
  • A model cites a scientific paper — convincing title, plausible authors, real-looking journal, valid-format DOI — but the paper does not exist
  • A model writes a Python function that compiles cleanly and looks correct, but produces wrong results for edge cases
  • A model describes a historical event with accurate-sounding dates and names but gets the sequence of events backwards
  • A model provides a detailed API method for a software library — but the method does not exist

Why Hallucinations Happen

Hallucinations are not primarily caused by a bug or an edge case. They are a natural consequence of what LLMs are built to do.

The Root Cause

LLMs predict the next token based on what is statistically likely given the context — not based on what is factually true. The model has learned what academic citations look like. When asked to produce one, it generates something that looks like a citation — because it has learned the pattern perfectly. Whether the cited paper actually exists is irrelevant to the prediction task. The model has no fact-checking mechanism.

Common Hallucination Patterns

01

Fabricated citations — papers, books, and articles with convincing titles, real-looking authors, plausible journals, and formatted DOIs that do not exist

02

Wrong attributions — correct quotes attributed to the wrong person; common with popular sayings that circulate without reliable sourcing

03

Invented people — fictional people with detailed biographical histories, publication lists, and professional credentials

04

False historical details — real events described with inaccurate dates, sequences, or participants, indistinguishable from accurate history without verification

05

Confident code errors — code that compiles cleanly but contains logic errors, incorrect API calls, or edge case failures not obvious without testing

06

Extrapolation from sparse data — fluent text about topics with little training data, where the model fills gaps with plausible-sounding but invented details

Why the Model Doesn't Know It's Wrong

No Separate Fact-Checking Layer

The same forward pass through the transformer produces a correct answer and a hallucinated one. The model has no access to ground truth, no database to query, no memory of what it verified vs. invented. From the model's perspective, generating a true fact and generating a hallucinated fact are the same computational operation — produce the statistically likely continuation. There is no internal alarm that fires when something is wrong.

Factors That Increase Hallucination Risk

Higher risk situations

Topics with sparse training data — rare events, niche domains, obscure people.

Very specific requests — exact dates, exact figures, exact quotes, specific citations.

High temperature or top-p settings — more exploratory sampling reaches lower-probability paths.

Long reasoning chains — errors compound across multi-step logic.

Lower risk situations

Summarization of a provided document — model works from in-context text, not from memory.

Well-represented topics — events and facts that appear extensively in training data.

Low temperature settings — fewer low-probability paths explored.

Short, simple factual questions about common knowledge.

Mitigation Strategies

01

Retrieval-Augmented Generation (RAG) — provide relevant documents in the context and have the model answer from them rather than from training memory

02

Ask for sources — prompt the model to provide citations. Makes hallucinations more detectable — a fake citation is checkable

03

Verification pipelines — pipe model output through a verification step (database lookup, another model, human reviewer) before delivering to users

04

Lower temperature — reduces exploratory sampling; lowers the rate of low-probability incorrect paths but does not eliminate hallucinations

05

Calibration prompting — explicitly instruct the model to say "I'm not certain" when unsure. Helps surface uncertainty but does not fully prevent confident errors

06

Task selection — hallucination risk varies by task. Route high-risk tasks (obscure facts from memory) to RAG pipelines; use direct model output only for lower-risk tasks

07

Human review for high-stakes domains — in medical, legal, and financial contexts, expert human review before acting on AI output is not optional; it is standard practice

The Honest Assessment

Not a Bug — A Property

Hallucinations will not be completely eliminated by the next model version. They are a property of systems that generate text by predicting likely continuations rather than retrieving verified facts. Progress is real — newer models hallucinate less, RLHF teaches better calibration, RAG reduces reliance on memory — but completely eliminating hallucinations while keeping generative flexibility is an unsolved problem. Treat LLM outputs like input from a very knowledgeable but sometimes overconfident colleague: valuable, but verify before acting.

Summary

Key Takeaways
  • Hallucinations are fluent, confident, factually incorrect outputs — the model sounds certain even when wrong
  • They happen because LLMs optimize for statistical likelihood, not factual accuracy
  • The model has no fact-checking mechanism — generating true and hallucinated content are the same operation
  • Common patterns include fabricated citations, wrong attributions, invented people, and false code
  • Risk is higher for rare topics, very specific facts, high temperature, and long reasoning chains
  • Mitigation includes RAG, asking for sources, verification pipelines, and mandatory human review for high-stakes domains
  • Hallucinations are a fundamental property of generative models, not just a temporary bug to be fixed

Need Help?

Ask the AI if you need help understanding hallucinations, why they happen, what types of tasks carry the highest hallucination risk, or how to design systems that reduce the impact of hallucinations on your users.