Why does an LLM make mistakes even when it seems confident? Because it doesn't know, it predicts.
It makes mistakes because it doesn't have an engine of truth inside it. It only has a statistical language engine. When it responds, it is not stating what is true: it is completing a sentence in the most probable way based on billions of examples.
An LLM does one thing: predicts the next word.
Every time you write something, it checks your text. It calculates which token--word, part of a word, punctuation--is most likely next. Then it writes it. And repeats. Token by token, until the response ends.
That's it. It doesn't reason or search for the right answer in real time. It already checked its knowledge base--billions of texts, web pages, books, articles--just once, during training. What it learned got compressed into statistical weights: not memories, not facts, not knowledge like humans have. If the sequence "the capital of France is" appeared thousands of times followed by "Paris," the model learned "Paris" is the most likely token there. When it responds, it's not querying anything--it's using what it absorbed.
The problem is this mechanism works even when it shouldn't. If enough text during training linked a writer to a wrong date, the model gives that date with the same confidence as the right one. It has no inner alarm. Just probabilities--and high probability doesn't mean true.
The difference with human error
When humans err, they often know they're unsure. They say "I think," "I'm not sure," "I'd check." Not always, but doubt is part of thinking.
An LLM has no doubt. Just probabilities.
Why understanding this matters
Not to stop using these tools. But to use them right.
An LLM is useful when treated as a fast collaborator to verify, not as an authoritative source. Blind trust is the issue--not the model itself.
A practical way to see if the model is in uncertain territory is to rephrase the same question two or three times differently. If answers change in details--a date shifts, a name varies, a number doesn't match--it means there's no fact but a spread of probabilities. The model picks the most likely token each time, and if the context shifts slightly, so does the pick.
You can also ask directly: "How sure are you about this answer? What are you basing it on?" Newer models can sometimes signal when they're in less-covered training territory--not always, not reliably, but often enough to know it's worth checking elsewhere. Same goes if you ask the model to think aloud: "Explain your reasoning step by step." Logical errors show up more easily when the model must lay out steps instead of jumping to conclusions.
