The Tool Desk
Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →Outbyte PC Repair FREEClear out junk files and repair common Windows errorsFree Scan →To build software with a large language model in 2026, you need eight working concepts: how generation works, tokens and context windows, prompting with examples, embeddings, retrieval-augmented generation (RAG), fine-tuning, tool calling with agent loops, and evaluation. This list is the organizing choice of this article, not a canonical syllabus that the title itself defines. It is written for developers who call hosted or open models from applications, not for anyone training a foundation model from scratch.
How the eight concepts fit together
The order below follows the sequence in which you meet these ideas when building an application: first what the model does with the text you send, then how you shape that text, then how you add outside knowledge, change behavior, let the model act, and finally check whether any of it works.
As an Amazon Associate I earn from qualifying purchases.
| # | Concept | The question it answers | Where it shows up in your app |
|---|---|---|---|
| 1 | LLM generation | What does the model actually produce? | The model call itself |
| 2 | Tokens and context windows | How much text fits in one request, and how is it counted? | Request size, cost, truncation |
| 3 | Prompting and examples | How do I steer the output without changing the model? | System and user messages |
| 4 | Embeddings | How do I find text that is similar in meaning? | Search and indexing layer |
| 5 | Retrieval-augmented generation (RAG) | How do I give the model material it was not trained on? | Retrieved context inserted into the prompt |
| 6 | Fine-tuning | How do I change the model’s learned behavior? | A separate training job and a new model artifact |
| 7 | Tool calling and agent loops | How does the model trigger actions outside itself? | Your code, which executes the tools |
| 8 | Evaluation | How do I know the system is good enough? | A test set and a repeatable scoring process |
1. How an LLM generates text
An LLM generates output by predicting the next token in a sequence, one step at a time, conditioned on the tokens it has been given. OpenAI’s API concepts documentation describes generation in these terms, and Google Cloud’s generative AI glossary defines the model’s output the same way. That single mechanism explains most of the behavior developers see: fluent answers, plausible-sounding errors, and sensitivity to wording.
What the model does not do by itself
A model call takes text in and returns text out. It does not browse the web, query your database, read your files, or send an email unless your application gives it a mechanism to do so. Microsoft Learn’s LLM fundamentals material describes tool use as a structured output that the model produces and that application code interprets and executes. Keeping this boundary in mind prevents a common design error: assuming that because the model “knows” something, it has checked it.
#1 Best Overall
2. Tokens and context windows
Models do not read characters or whole words. They read tokens, which are chunks of text that may be a whole word, part of a word, punctuation, or whitespace. Longer or unusual words, identifiers, and non-English text often split into more tokens than common English words do.
Counting tokens
OpenAI’s key concepts page gives a rough rule of thumb for English: about 4 characters or 0.75 words per token. Treat that as an approximation for estimating budgets, not a conversion you can rely on. The exact count depends on the tokenizer used by the specific model, and it changes across languages and content types. If you need exact numbers, use the tokenizer or token-counting tool that your provider documents for the model you call.
The context window
The context window is the maximum amount of tokenized material a model can handle in one request. It covers what you send, including instructions, conversation history, retrieved documents, and tool results, and it also covers the output the model generates. Practically, that means a long prompt leaves less room for the answer, and a request can fail or be truncated if it exceeds the limit. Context limits differ by model and change over time, so there is no single number to memorize. Check the documentation for the exact model you deploy.
Quick wins for a faster PC:
Repair Windows errors before they cause bigger problemsFix Now →Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →3. Prompting and examples
A prompt is the set of instructions and context you send with a request. Good prompts state the task, the constraints, the output format, and the relevant background. Where the task has a specific pattern, you can include a few worked examples in the prompt. This is called few-shot prompting, and it demonstrates the input-output pattern you want without changing the model’s weights.
What prompting can and cannot do
Prompting is a way of shaping the request. It can make output more consistent, more structured, and closer to the style you need. It cannot guarantee correctness, and it cannot add knowledge the model lacks unless that knowledge is included in the prompt itself. When a prompt keeps failing in the same way, that is a signal to measure the failure (see evaluation below) before rewriting it again.
4. Embeddings
An embedding is a vector, a list of numbers, that represents a piece of data so that its meaning or content is reflected in its position relative to other vectors. Items with similar content tend to have nearby embeddings. That property powers semantic search, clustering, recommendation, and classification.
What similarity does and does not tell you
Embedding similarity is good at finding candidates: passages that are probably about the same topic as a query. It does not establish that a passage is true, current, or the right passage to answer a specific question. A passage about the same product can be about the wrong version. Treat embedding search as a ranking step whose output still needs checking.
5. Retrieval-augmented generation (RAG)
RAG adds relevant external information to the model’s context at the moment of the request. The model’s weights do not change. The application finds material that is likely to help, inserts it into the prompt, and asks the model to answer using that material. OpenAI’s prompt engineering guidance presents retrieval as one way to add context, alongside prompting and fine-tuning as methods for improving accuracy.
Rank #3
How a basic RAG pipeline works
- Split your source documents into chunks that are small enough to be useful and large enough to keep their meaning.
- Compute an embedding for each chunk and store it in a vector index or search store.
- When a user asks a question, embed the question and retrieve the most similar chunks, usually a fixed number such as the top few results.
- Insert the retrieved chunks, along with the question and clear instructions, into the prompt.
- Ask the model to answer from the supplied material and to say when the material does not contain the answer.
Where RAG fails
- Retrieval misses. The right chunk is never retrieved, so the model answers from general knowledge or guesses.
- Irrelevant context. Loosely similar chunks dilute the prompt and can pull the answer off course.
- Stale indexes. The source changed but the stored embeddings did not, so the answer reflects an outdated version.
- Context overflow. Too many retrieved chunks push the request toward the context window limit.
RAG improves grounding, but it is not an automatic correctness guarantee. Each of these failures needs its own check.
6. Fine-tuning
Fine-tuning continues training a model on additional examples so that its learned behavior changes. The result is a different model, or an adapted version of one, that tends to respond in the trained style or follow a trained pattern without needing as much instruction in every request.
Fine-tuning is not the same as adding information
Prompting and RAG supply context at request time. Fine-tuning adapts what the model has learned. That distinction matters in practice. Fine-tuning is not the usual way to make a model aware of documents that change weekly, because those documents would need retraining each time. It is also not always necessary: OpenAI’s accuracy guidance recommends first diagnosing where a system fails, then choosing an intervention that addresses that failure. Some failures are fixed by a better prompt, some by retrieval, and some, such as consistent adherence to a specialized output style, may call for fine-tuning.
Do these 3 things before closing this tab:
1Scan for outdated or missing drivers - takes under a minute2Clear out junk files and repair common Windows errors3Fix the driver behind crashes, sound loss and screen glitchesPrompting, RAG and fine-tuning compared
These three approaches are often confused because all of them change the output. They operate on different parts of the system, so they address different failures.
| Approach | What it changes | When the change takes effect | Good fit when | Main limitation |
|---|---|---|---|---|
| Prompting and examples | The instructions and demonstrations in each request | Per request | The model has the capability but needs direction or format guidance | Cannot add knowledge that is absent from the prompt |
| RAG | The retrieved context supplied per request | Per request, based on what retrieval returns | Answers depend on your documents or data that change over time | Output quality depends on retrieval quality |
| Fine-tuning | The model’s learned behavior | After a training job produces a new model | A consistent behavior or pattern must be learned rather than repeated in every prompt | Requires training data and a retraining cycle; not a way to add live facts |
The sources reviewed for this article describe these as different approaches and do not provide a universal ranking or a threshold that tells you when to switch. The deciding factor is the failure you have measured.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.7. Tool calling and agent loops
Tool calling lets the model request that a function be run. You describe each tool to the model, typically with a name, a description, and a parameter schema. When the model decides a tool is needed, it returns a structured request instead of a plain answer. Your code does the rest.
The tool call cycle
- Define each tool with a clear name, a description of when to use it, and a schema for its parameters.
- Send the user request along with the tool definitions.
- Read the model’s response. If it contains a tool request, parse the name and arguments.
- Validate the arguments, then execute the function in your own code.
- Return the result to the model as a new message, and let it produce either a final answer or another tool request.
Validation in step 3 and 4 matters because model-produced arguments can be malformed, out of range, or inappropriate for the user’s permissions. Your code should reject or confirm them rather than trust them.
Free tools Windows power users keep installed
One-click scans. No signup required.
Agent loops
An agent repeats this cycle: model reasons, requests a tool, observes the result, and continues until it decides the task is done. The loop is simple to write and easy to make unbounded. Put limits on it. Cap the number of iterations, set timeouts on tool calls, and require explicit confirmation before any action with side effects, such as deleting data, sending messages, or spending money. The model proposes; your system decides what actually runs.
Best Value
8. Evaluation
Evaluation means checking how the model and your application behave on tasks that represent real use, against criteria you have written down. Without it, you cannot tell whether a prompt change helped, whether retrieval is returning the right chunks, or whether a fine-tuned model is better than the base model.
Build a representative test set
Collect realistic inputs from your domain, including awkward and edge cases, and record what a correct or acceptable output looks like. For retrieval, record which documents should be found. Keep the set stable enough to compare runs, and add cases whenever you find a new failure in production.
Diagnose before you intervene
OpenAI’s accuracy guidance frames the process as diagnosis followed by intervention. A workable sequence looks like this:
- Run the test set and label each failure: wrong format, missing information, wrong information, or inappropriate action.
- Check the retrieval output for failures where the model was given no usable material.
- Rewrite the prompt or add examples for failures about format, tone, or instruction-following.
- Improve chunking, retrieval, or the supplied context for failures caused by missing or irrelevant material.
- Consider fine-tuning only for consistent behavioral failures that persist after the earlier fixes, then re-run the same test set to compare.
The acceptable error rate depends on the use case and on what a wrong answer costs. A drafting assistant and a system that gives medical or financial guidance need very different bars. Be cautious with any published accuracy score that does not state the task, dataset, and conditions behind it.
Model behavior, context limits, and API features change over time. Confirm specifics against the current documentation for the exact model you deploy.
Quick Recap
Further reading
- OpenAI API documentation: “Key concepts” (tokenization, context limits, embeddings, generation), “Prompt engineering” (examples, retrieval, context), and “Optimizing LLM Accuracy” (evaluation and choosing between prompting, RAG and fine-tuning).
- Microsoft Learn: “LLM Fundamentals,” covering next-token generation and application-side tool execution.
- Google Cloud: “Generative AI glossary,” with definitions of RAG, tokenization, context windows, and function calling.
- Hands-On Large Language Models, a book covering foundations, RAG, fine-tuning, vector databases, and evaluation for a longer hands-on treatment. Check the publisher for the current edition before buying.
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
