PC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11Crashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minuteAn LLM generates a response by processing text as tokens, scoring possible next tokens, selecting one, and repeating that cycle until it reaches a stopping condition or output limit. In a deployed assistant, the surrounding software may also add instructions, retrieved information, or tool results before or during generation.
What happens after you send a prompt to an LLM?
- The system assembles the context. The model may receive your prompt along with earlier conversation, instructions, and other relevant input. An assistant application can also supply retrieved material or results from tools; those are additions around the model, not the same thing as its basic text-generation loop. OpenAI’s tool documentation describes how tools can be used in an application.
- A tokenizer turns text into tokens. Tokens can be whole words, parts of words, or punctuation. Their boundaries depend on the tokenizer and model, so a token is not simply another name for a word. OpenAI gives a rough English estimate of about four characters or three-quarters of a word per token, but actual counts vary by encoding and language. OpenAI explains tokens and counting.
- The network processes the token context. In a common Transformer architecture, layers update token representations. Self-attention lets the model use relationships among tokens in the available context when processing them. This is a high-level account of Transformer-based models, not a claim that every LLM uses an identical design. The original Transformer paper describes the architecture.
- The model scores possible continuations. At each generation step, the model produces scores or a probability distribution for possible next tokens. A decoding method uses those scores to select a token. Depending on the method and settings, the system need not choose the single highest-scoring option every time. OpenAI’s text-generation guide describes the generation process.
- The selected token is added to the context. The system repeats the scoring and selection loop, using the growing sequence to generate what comes next.
- Generation stops. It ends when the model or application reaches a stop condition, such as an end token, or when an output limit is reached. The prompt and generated output must fit within the applicable model’s context limit; the specific limit varies by model. The text-generation guide covers the overall process.
Does an LLM predict one word at a time?
“It predicts the next word” is a useful shorthand, but it is incomplete. For a common autoregressive text generator, each step predicts a next token, which might be a word, a word fragment, or punctuation. The model processes context to score possible continuations, and a decoding method selects among them. The selected token is then fed into the next step.
As an Amazon Associate I earn from qualifying purchases.
This describes the repeated output process, not everything the model learned during training or every part of an assistant product. Retrieval, tool calls, instructions, and other application logic can affect the context or what happens around generation.
Do these 3 things before closing this tab:
1Scan for outdated or missing drivers - takes under a minute2Repair Windows errors before they cause bigger problems3Fix the driver behind crashes, sound loss and screen glitchesWhy can the same prompt produce different answers?
The model can assign plausible scores to more than one continuation. A decoding method may always select the top-scoring token, or it may sample from plausible options; the selection method and generation settings affect the result. So “the model always picks the most likely word” is not a reliable description of every system.
#1 Best Overall
Even when the model’s next-token process is the same, a changed conversation context or added information can change the available continuations. A different answer does not, by itself, mean the system looked up a different finished response.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.What is a token in an LLM?
A token is a unit used to represent text for a model. Depending on the tokenizer, it can correspond to a whole word, part of a word, punctuation, or another text segment. The same sentence can therefore have different token counts under different encodings, and token counts can vary across languages. OpenAI’s estimate of roughly four English characters or 0.75 words per token is only a rule of thumb, not a conversion formula. See OpenAI’s token guidance.
Quick Recap
Best Value
What the basic generation loop does—and does not—explain
- It explains how text can be produced incrementally: process context, score candidate next tokens, select a token, and repeat.
- It does not mean every LLM has the same architecture: the Transformer account applies to a common model family, not every system.
- It does not describe every assistant feature: a deployed assistant may use retrieval or tools in addition to model generation.
- It does not guarantee a fixed response: decoding choices and context can affect which continuation is produced.
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
