Free tools Windows power users keep installed
One-click scans. No signup required.
An LLM’s context window is a request budget, and it covers much more than the question you typed. When a request is too large, the platform may reject it, cut off the output, or change what the model sees. Even when everything fits, the model may not reliably use the detail you need. Good context engineering handles two different problems: a capacity failure, where the input or output does not fit, and a context-use failure, where the material fits but the model does not find or apply it.
What context engineering means
Microsoft’s documentation defines the practice as “deliberately managing what information an AI model can see when processing a request.” The working unit is the context for one request: the instructions, history, source material, and tool results that the model receives at that moment. It is not everything the model learned in training, and it is not the entire history of an application.
As an Amazon Associate I earn from qualifying purchases.
What counts toward the window
In OpenAI’s API, context-window accounting includes input tokens, output tokens, and, for some models, reasoning tokens. Generated output therefore draws on the same budget as the prompt. A long answer leaves less room for input, and a request that looks short can still be large once reasoning and output are counted.
In agentic tools, the assembled context usually contains more than the prompt you wrote. In Visual Studio Code, for example, an agent request can draw on:
#1 Best Overall
- built-in instructions and customization files,
- the current user message,
- earlier chat history,
- the active file or editor state,
- files you reference explicitly, and
- output returned by tools.
Explicit references consume context space, so attaching a file helps only when that file bears on the current task. Exact accounting differs across products and endpoints, and a token count reported by one tool does not transfer to another.
What happens when the window is full
The outcome depends on the platform and its version. There is no universal rule that the model silently deletes the oldest messages, and you should check the behavior of the specific product you use.
API requests: rejection or truncated output
OpenAI warns that exceeding the allocated window may result in truncated outputs, and it says that generated tokens beyond the limit may be truncated in API responses. The practical risk is an incomplete answer that looks finished. Check whether a response stopped because it hit a length limit before you rely on it, and treat a truncated structured output (such as JSON) as a failed call rather than a result to parse.
Recommended Free Tools
Chat products: rolling history
Some chat products roll history forward, so earlier turns can drop out of the working context as the conversation grows. Which turns stay, and how the product signals that change, varies by product and version. If an answer ignores a constraint you gave early in a session, restate it in the current message.
Compaction: a summary replaces earlier state
Compaction condenses earlier interaction state into a summary. OpenAI offers compaction in its Responses API, configured with context_management and compact_threshold, and also provides a standalone compact endpoint. Anthropic documents server-side compaction for long-running workflows. Availability and parameter names can change, so confirm them in the current documentation before building on them.
A summary is only as good as what it keeps. It may omit a number, a rejected option, or a caveat that the next step depends on. Compaction is a reasonable tool for long sessions, but it does not preserve every detail.
Fitting is not the same as being used
The clearest evidence on this point comes from Liu and colleagues in Lost in the Middle: How Language Models Use Long Contexts, published in Transactions of the Association for Computational Linguistics in 2024 after a 2023 preprint. The authors tested multi-document question answering and key-value retrieval. Their abstract states: “performance is often highest when relevant information occurs at the beginning or end of the input context, and significantly degrades when models must access relevant information in the middle of long contexts, even for explicitly long-context models.”
Outdated Drivers Are Slowing You Down
One free scan finds every outdated or missing driver and matches the right update for your exact hardware.Free scan · exact hardware matchWindows Errors? Fix Them Before They Spread
Repair common Windows errors and clear accumulated junk for a smoother, more stable PC - no reinstall needed.Free scan · no reinstallThat finding describes the tasks and systems the authors evaluated. It does not show that every current model ignores material in the middle of its input. It does show that a relevant passage can be inside the window and still be used poorly, which is why capacity alone is not a quality measure. When a long input contains one critical fact, test whether moving that fact to the beginning or end changes the answer.
Does a bigger window make answers more accurate?
Not automatically. Anthropic states that larger windows do not automatically make more context better, and recommends curating what goes in. Google’s guidance for Gemini notes that multi-needle retrieval, where a model must find several facts at once, can be less accurate than a single-needle test. Google says many Gemini models have context windows of 1 million or more tokens. That is a statement about capacity for those models, not a guarantee of accuracy across a long input. Google also notes that longer queries generally have higher time-to-first-token latency.
Remove material that does not help the task. Extra files, old turns, and verbose tool output can dilute the parts that matter, and they add cost and latency whether or not they improve the answer.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Retrieval, caching, and compaction solve different problems
These three approaches are often grouped together, but they answer different needs. Retrieval selects external material and places it into a request. Caching lets the same context be reused across requests. Compaction condenses earlier state in a long-running conversation. Compare them on the axes that matter for your task:
| Axis | Retrieval | Caching | Compaction |
|---|---|---|---|
| Coverage and recall | Depends on whether the retriever returns the needed evidence; check this on your own queries | No change to what is included; it reuses the same context | Limited to what the summary preserves |
| Position sensitivity | Depends on where selected passages are placed; the reviewed evidence does not give a general rule for every model | Not stated in the reviewed sources | The summary’s own placement is not a documented factor in the reviewed sources |
| Latency | Adds a retrieval step; no general figure stated in the reviewed sources | Google describes caching for repeated context; no latency figure is stated for caching in the reviewed sources | Reduces the amount of prior state sent; no general figure stated |
| Token and storage cost | Sends only selected material; storage depends on the index and vendor | Depends on provider caching terms and current pricing, which should be checked directly | Summarization itself consumes tokens; savings depend on the session |
| Implementation complexity | Highest: requires indexing, chunking, and retrieval evaluation | Moderate: depends on provider API support | Moderate: OpenAI and Anthropic each document their own configuration |
| State fidelity after summarization | Not applicable: original passages are retrieved | Not applicable: content is not summarized | A summary may omit details; keep critical facts in an explicit record |
| Provider-specific limits | Vary by vendor and tooling | Vary by provider | OpenAI: Responses API context_management and compact_threshold, plus a standalone compact endpoint; Anthropic: server-side compaction |
The reviewed sources do not establish a safe percentage of a window to use, and they do not identify one ordering strategy that works across all providers and tasks.
A practical procedure
- Define the output. Write down what the model must answer or produce. Anything that does not serve that output is a candidate for removal.
- Separate durable instructions from the current request. Keep standing rules in one place, and keep the current question short and explicit.
- Include only history and sources that bear on the task. Remove old turns, unrelated files, and verbose tool output when they do not help.
- Retrieve instead of injecting large corpora. Then check that the retrieved passages contain the evidence the answer needs. Retrieval that returns the wrong passages gives the model a confident-looking but incomplete basis.
- Test placement for critical facts. For long inputs, move the one or two facts the answer depends on to the beginning or end and see whether the result changes.
- Compare caching options if the same large context repeats. Check the provider’s current caching terms, pricing, and latency rather than assuming savings.
- For long sessions, compact or reset deliberately. Keep decisions, constraints, and key figures in an explicit record outside the summary, so a summary cannot silently remove them.
- Evaluate the assembled prompt on representative tasks. Judge the output, not the advertised maximum window.
What the evidence does not establish
- No universal token count marks the point where answer quality falls. Quality depends on the model, the task, and the arrangement of the input.
- No safe percentage of a window is established by the reviewed sources.
- Google’s statement that many Gemini models have windows of 1 million or more tokens describes capacity for those models at the time of its documentation; check the specific model page before relying on a figure.
- The survey A Survey of Context Engineering for Large Language Models (2025) reports analyzing more than 1,400 papers. That is the authors’ own count and has not been independently verified.
- The position effect in Lost in the Middle was measured on question answering and key-value retrieval. Results on other tasks, and on newer models, may differ.
Context engineering is therefore less a fixed rule than a discipline: decide what the task needs, confirm what the platform will actually do with it, and verify the answer.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
