🤖

Series · Part 6 of 21

Foundation
AI Demystified
Abhishek Saha
Abhishek Saha
· 🤖 AI / ML

Why AI Lies (And Doesn't Know It)

AI sometimes states false things with complete confidence — fake citations, wrong numbers, invented APIs. This is hallucination. Here's why it happens and how to catch it.

You ask AI a question. It answers immediately, in full sentences, with specific details. It sounds authoritative.

And it’s completely wrong.

This is hallucination — and it’s not a bug that will eventually be fixed. It’s a fundamental consequence of how language models work.

HALLUCINATION— why AI confidently lies

These are real-style responses an AI might give. Each one sounds confident. Spot the problem, then reveal why it happens.

YOU ASKED
Can you cite a research paper about AI and creativity?
AI RESPONDED
Sure! "The Creative Frontier: Machine Intelligence and Artistic Expression" by Dr. Sarah Chen and Prof. James Liu, published in Nature Machine Intelligence, Vol. 12, Issue 4, pp. 234–251 (2023). DOI: 10.1038/s42256-023-00445-2

The Core Reason

The model was trained to predict plausible next tokens. It’s extraordinarily good at that — but plausibility is not the same as truth.

When you ask for a research citation, the model knows that citations have: a title, authors, a journal name, a volume, an issue number, and a DOI. It generates a plausible example of each. If a paper matching that description doesn’t exist, the model doesn’t know — because it doesn’t check. It generates.

Why the Confidence Feels Convincing

The same mechanisms that make the model’s writing fluent also make its false statements fluent. Hedged, uncertain language (“I think maybe…”) was less common in training data than confident, declarative statements. The model learned to write like an expert — even when it doesn’t have expert knowledge.

What Actually Helps

Retrieval-Augmented Generation (RAG) — where the model can search the web or a database before responding — dramatically reduces hallucination on factual questions. This is what powers Perplexity, ChatGPT’s web mode, and Claude’s document analysis.

When retrieval isn’t available, the model is working entirely from memory. And that memory has gaps it doesn’t know about.

The Right Mental Model

Think of AI like a very well-read person who has read millions of books but cannot cite their sources, cannot verify their recollections, and doesn’t know what they don’t know. When they’re confident, they’re fluent. When they’re wrong, they’re equally fluent.

Use it as a brilliant first draft and thinking partner. Verify anything that matters.

Next up: Part 7 goes deeper on AI’s structural limits — not just hallucination, but the fundamental things no current AI system can do, and why those limits exist.

AI Demystified · 16 of 21 published

  1. 0 Grounding 5 Mental Models You Need Before Diving Into AI
  2. 1 Foundation What Happens When You Ask AI Something?
  3. 2 Foundation Transformers — The Architecture That Changed Everything
  4. 3 Foundation How AI Learns, Thinks, and Decides
  5. 4 Foundation How AI Reads Your Words
  6. 5 Foundation Why AI Forgets
  7. 6 Foundation Why AI Lies (And Doesn't Know It)
  8. 7 Foundation What AI Cannot Do
  9. 8 Foundation How AI Reasons (And Why It Sometimes Breaks)
  10. 9 Practice Prompt Engineering — How to Talk to AI
  11. 10 Practice Embeddings & Vector Databases — The Memory Layer of AI
  12. 11 Practice RAG Explained — How AI Knows What You Didn't Train It On
  13. 12 Practice Fine-tuning vs. Prompting — When to Use Which
  14. 13 Practice Do You Really Need GPT-4?
  15. 14 Practice Latency, Tokens, and Cost — The Physics of AI Products
  16. 15 Practice How Do You Know AI Is Actually Working?
  17. 16 Hands-On Coding Setup — Your AI Development Environment soon
  18. 17 Hands-On MCP Tool Calling — How AI Uses Tools soon
  19. 18 Hands-On AI Agents — Beyond Chatbots soon
  20. 19 Hands-On Build Your First Real AI App soon
  21. 20 Hands-On Token Optimization — Spend Less, Get More soon

Related posts

Every Team Built an AI Agent. Nobody Knew Who Used Which Model, or What It Cost.
AI / ML ·

Part 1 — Building an Agent Platform

Every Team Built an AI Agent. Nobody Knew Who Used Which Model, or What It Cost.

One gateway in front of every agent: role-based access, human approvals, and every LLM call logged per user, agent and model.

Read
An Enterprise Agent Platform on My Laptop, in 8 Containers
AI / ML ·

Part 2 — Building an Agent Platform

An Enterprise Agent Platform on My Laptop, in 8 Containers

The agent platform on a laptop, screen by screen: start the stack, deploy a Langflow and a LangChain agent, wire up models, chat, approve.

Read
Enterprise Master Data Cleaning: The Business Case for SAP Supplier Data
AI / ML ·

Part 1 — Enterprise Master Data Cleaning

Enterprise Master Data Cleaning: The Business Case for SAP Supplier Data

Why dirty vendor master data quietly costs enterprises millions, how AI helps, and the operating model that makes it safe. Part 1 of the series, for business and leadership.

Read
newsletter

Get new posts in your inbox

No spam. No digest. Just a note when I publish something new.

Discussion