What is an LLM? Tokens, context, and why the AI forgets

An LLM, a large language model, is the machine inside ChatGPT, Claude, Gemini, and every AI builder you will ever touch. It is a text-prediction engine trained on a huge slice of everything humans have written, and its one trick, predicting what text should come next, turns out to cover writing, planning, and code.

You do not need to know how it works inside. You need three words: token, context window, and hallucination. Those three explain almost every strange thing your AI tools do, including the most confusing one, why the AI that built half your app this morning has forgotten it by the afternoon.

What does 'predicting text' actually mean?

An LLM reads everything you have given it and asks one question, billions of times per answer: given all of this, what is the most likely next little piece of text? It writes your landing page the same way it finishes the sentence 'the cat sat on the...': by prediction, not by looking facts up in a database.

That is its superpower and its flaw in one. Prediction is why it can write code, a sales email, and a privacy policy with equal confidence. It is also why the confidence never wavers even when the content is wrong: the machine is not checking truth, it is continuing a pattern.

What is a token?

A token is the unit an LLM actually reads and writes: a chunk of a word, roughly three-quarters of one on average. 'Booking' might be one token; 'unbelievable' might be three. Every AI tool you use is counting them constantly, because tokens are how the work is measured.

• Pricing is per token. When a tool bills by usage or credits, long conversations and big pastes are what you are paying for.
• Limits are in tokens. 'Message too long' means you have hit a token ceiling, not a character count.
• Speed follows tokens. The model writes its answer one token at a time, which is why replies stream out like a typist.

What is a context window, and why does the AI forget?

The context window is the model's working memory: the maximum number of tokens it can consider at once. Everything has to fit inside it, your instructions, the conversation so far, the code it wrote earlier. It is the desk the model works at, and the desk has edges.

Here is the part that explains the forgetting: the model has no memory between or beyond that desk. When a conversation grows past the window, the oldest material falls off the edge, and the AI genuinely no longer knows what it said two hours ago. It did not get lazy. The instructions simply left the room.

The practical habits follow directly. Start a fresh conversation for each new task instead of one endless thread. Restate the important facts when you switch topics. Keep a short project brief, what the product is, who it is for, what is built so far, and paste it at the top of new sessions. Founders who do this stop feeling gaslit by their tools.

Why does it make things up?

Because prediction always produces an answer, whether or not a good one exists. When the model does not know, it does not say 'no idea'; it continues the pattern with something plausible. That is a hallucination: fluent, confident, and wrong. It invents statistics, legal clauses, and occasionally features your app does not have.

The defense is cheap: treat the AI like a brilliant intern on their first week. Let it draft everything, and verify the things that have consequences, names, numbers, prices, claims about the law. For code, the same instinct becomes a habit you will meet in the Build stage: trust, then check.

The words 'token' and 'context window' now live in the Glossary too, for the next time a tool throws them at you mid-build.

Common questions