Series · Part 3 of 21
FoundationHow AI Learns, Thinks, and Decides
Training, inference, sampling, fine-tuning — these words are everywhere. Here's what they actually mean, with live visualizations and honest analogies.
You’ve seen the pipeline — tokens in, words out. But how does the model know anything in the first place? How did it get good at this? And why does it sometimes give different answers to the same question?
These four concepts explain everything.
The Four Things Worth Understanding
Training is how the model becomes capable. It reads text, predicts what comes next, gets told how wrong it was, and adjusts. Trillions of times. After enough repetitions, it stops being wrong.
Inference is what happens when you actually use it. The model runs your prompt through all its layers — once per token it generates — and produces output left to right. Every word you read was predicted one at a time.
Sampling is why the same prompt doesn’t always give the same answer. The model doesn’t just pick the most likely word every time — it samples from a probability distribution. Temperature controls how adventurous that sampling is.
Fine-tuning is how a general model becomes a specialist. The base model knows everything. Fine-tuning teaches it to behave a particular way — as a doctor, a coder, or a polite customer support agent.
Why Sampling Matters Most (For You)
If you use AI tools regularly, temperature is the one setting that changes your experience most:
- Low temperature → predictable, consistent, good for factual tasks
- Medium temperature → balanced, good for most writing and coding
- High temperature → creative, surprising, sometimes wrong
Most tools don’t expose this directly. But knowing it exists helps you understand why AI sometimes surprises you.
Next up: Before we dive deeper into how AI stores knowledge, let’s go one level down. How does the model even read your words? The answer is tokens — and it changes everything about how you think about AI input.
AI Demystified · 16 of 21 published
- 0 Grounding 5 Mental Models You Need Before Diving Into AI
- 1 Foundation What Happens When You Ask AI Something?
- 2 Foundation Transformers — The Architecture That Changed Everything
- 3 Foundation How AI Learns, Thinks, and Decides
- 4 Foundation How AI Reads Your Words
- 5 Foundation Why AI Forgets
- 6 Foundation Why AI Lies (And Doesn't Know It)
- 7 Foundation What AI Cannot Do
- 8 Foundation How AI Reasons (And Why It Sometimes Breaks)
- 9 Practice Prompt Engineering — How to Talk to AI
- 10 Practice Embeddings & Vector Databases — The Memory Layer of AI
- 11 Practice RAG Explained — How AI Knows What You Didn't Train It On
- 12 Practice Fine-tuning vs. Prompting — When to Use Which
- 13 Practice Do You Really Need GPT-4?
- 14 Practice Latency, Tokens, and Cost — The Physics of AI Products
- 15 Practice How Do You Know AI Is Actually Working?
- 16 Hands-On Coding Setup — Your AI Development Environment soon
- 17 Hands-On MCP Tool Calling — How AI Uses Tools soon
- 18 Hands-On AI Agents — Beyond Chatbots soon
- 19 Hands-On Build Your First Real AI App soon
- 20 Hands-On Token Optimization — Spend Less, Get More soon
Related posts
Part 1 — Building an Agent Platform
Every Team Built an AI Agent. Nobody Knew Who Used Which Model, or What It Cost.
One gateway in front of every agent: role-based access, human approvals, and every LLM call logged per user, agent and model.
ReadPart 2 — Building an Agent Platform
An Enterprise Agent Platform on My Laptop, in 8 Containers
The agent platform on a laptop, screen by screen: start the stack, deploy a Langflow and a LangChain agent, wire up models, chat, approve.
ReadPart 1 — Enterprise Master Data Cleaning
Enterprise Master Data Cleaning: The Business Case for SAP Supplier Data
Why dirty vendor master data quietly costs enterprises millions, how AI helps, and the operating model that makes it safe. Part 1 of the series, for business and leadership.
ReadGet new posts in your inbox
No spam. No digest. Just a note when I publish something new.
Discussion
On this page