Model distillation is a way to train one AI model to imitate another. A larger “teacher” supplies learning signals—such as its output probabilities, internal representations or generated answers—to a “student” model. By contrast, ordinary AI use usually means inference: giving an already-trained model an input and receiving an output. Distillation creates or updates a model; prompting uses one that has already been trained.
What happens during model distillation?
A distillation workflow uses a teacher model to help train a student for a particular task. The student may be smaller, but size alone does not establish that it will perform as well as the teacher.
As an Amazon Associate I earn from qualifying purchases.
- Choose the teacher and student. Select the model whose behavior will guide training and the student model or training setup to adapt.
- Prepare relevant prompts or examples. The training material should reflect the student’s intended use.
- Collect a learning signal from the teacher. Depending on the method and what access is available, this may be output probabilities, generated responses or intermediate representations.
- Train the student against that signal. The student learns to approximate the selected behavior. Some methods also ask the teacher to assess sequences the student generated itself.
- Evaluate the student on held-out, task-relevant data. Check it under the conditions in which it will be deployed; a handful of plausible answers does not prove equivalence to the teacher.
The UK Government’s AI Insights guidance on model distillation, updated August 3, 2026, describes the technique as model compression. Amazon Bedrock provides one managed example: its model-distillation documentation describes selecting teacher and student models, using prompts or invocation logs to generate teacher responses, and fine-tuning the student. A cloud service is one implementation, not a requirement for distillation.
Quick wins for a faster PC:
Clear out junk files and repair common Windows errorsFree Scan →Scan for outdated or missing drivers - takes under a minuteDriver Scan →Repair Windows errors before they cause bigger problemsFix Now →Distillation versus ordinary AI use
| Ordinary AI use (inference) | Model distillation |
|---|---|
| You send an input to an already-trained model and use its output. | A teacher’s behavior supplies a training signal for a student model. |
| The request uses the model; it does not, by itself, replace its parameters with a newly trained student. | Training produces or updates the student, which can then be used for inference. |
| Usually happens as a per-request activity. | Includes a training stage, such as generating or collecting data and fitting the student, followed by evaluation. |
| Compute is needed to serve the request. | There is additional work and compute for data generation and training; any later serving savings depend on the resulting student and deployment. |
An analogy: ordinary use is asking a knowledgeable system a question; distillation is training another system using examples or other signals from how the first behaves. The analogy has limits: distillation can transfer probability distributions or internal representations, not just visible answers.
#1 Best Overall
What can a teacher pass to a student?
Output probabilities and soft targets
Response-based methods can train a student against the teacher’s output distribution—the relative probabilities assigned to possible outputs—rather than only against a single hard label. That distribution can convey uncertainty and relationships between alternatives. The UK Government guidance describes this kind of signal; the precise information available depends on access to the teacher.
Intermediate features
Feature-based distillation uses internal representations, such as hidden activations, as part of the training signal. This differs from training only on the teacher’s final answer and requires a method that can access or otherwise use those representations.
Teacher-generated responses
A teacher can generate prompt–response examples that are then used to fine-tune a student. This synthetic-data approach is common in documented workflows, but it is not identical to every form of distillation: for example, it need not train directly against the teacher’s full probability distribution.
Do these 3 things before closing this tab:
1Repair Windows errors before they cause bigger problems2Scan for outdated or missing drivers - takes under a minute3Clear out junk files and repair common Windows errorsSelf-distillation and student-generated sequences
Distillation does not always require a separately chosen external teacher. In self-distillation, later checkpoints or deeper parts of a model can supervise earlier checkpoints or shallower parts.
Rank #3
- Incredibly Light. Surprisingly Thin. - LG gram is designed to go wherever you do. Weighing just 2.5 lbs. with an ultra-slim 0.7-inch profile, it slips easily into your bag and feels light in hand—making it effortless to carry, commute, and work from anywhere.
- Remarkably Light. Reliably Strong. - LG gram has passed seven military-grade durability tests, striking an impressive balance between a highly portable, lightweight metal build and the confidence to handle everyday movement and travel.
- Power That Last with Smart Efficiency - LG gram combines a high-capacity 72Wh battery with AI-driven power management to optimize efficiency based on your usage. The result is up to 32 hours of video playback for} long-lasting performance that keeps up with your day—at home, at work, or wherever you go.
- AMD Ryzen AI Performance - Powered by AMD’s AI-optimized Ryzen processor with Radeon Graphics and a built-in NPU, LG gram delivers smooth multitasking and responsive performance. Fast 32GB LPDDR5x memory and 1TB NVMe storage keep everything moving without slowdowns.
- Dual AI for Always-On Intelligence - LG gram’s Dual AI—powered by EXAONE 3.5, LG’s AI solution—combines gram chat On-Device AI and gram chat Cloud AI to deliver seamless assistance. gram chat On-Device AI enables fast document search and summarization directly on your PC, while gram chat Cloud AI expands capabilities when connected—so everyday tasks stay smooth, responsive, and uninterrupted.
There is also a mismatch that can arise when a student is trained only on fixed sequences but later generates its own sequences at deployment. Google DeepMind’s 2024 work on on-policy distillation of language models studies training on student-generated sequences with teacher feedback. This approach is intended to address that mismatch; it does not make evaluation unnecessary.
Why distill a model—and what can go wrong?
The aim is often to make a model less costly or faster to serve, reduce memory requirements, or make deployment on constrained hardware more practical while retaining enough performance for a defined task. Those are possible benefits, not automatic results. A student may be smaller yet fail to match the teacher where it matters.
The UK Government’s August 2026 guidance gives illustrative figures: it says students may retain 80% to 95% of a teacher’s task-specific quality and cites an 8-billion-parameter student responding in under 100 milliseconds on a single accelerator versus a 70-billion-parameter teacher taking several seconds and potentially requiring multiple GPUs. It also describes 80% to 95% fewer compute resources as a general claim. These are figures reported by the guidance, not guaranteed outcomes or a universal benchmark for a particular workload.
Research explains why results vary. A NeurIPS 2021 study by Samuel Stanton and co-authors found that the distillation dataset and temperature scaling affect how closely student and teacher predictive distributions match, and that substantial discrepancies can remain even when the student has capacity to match the teacher. A DistiLLM paper at ICML 2024 reported up to 4.3× speedup over recent knowledge-distillation methods in its evaluated setup; that is a result for that paper’s experiments, not a general speedup for distilled models. A 2024 preprint studying Llama 3.1 405B as teacher and 8B/70B students likewise emphasizes synthetic-data quality and task-specific evaluation; its findings apply to the models, tasks and data it tested.
Best Value
How to judge whether a distilled model is suitable
Compare it with the teacher on the task the student is meant to perform, and measure both quality and practical deployment outcomes. Useful checks include:
- Task performance: Evaluate on representative held-out examples, including important edge cases.
- Deployment behavior: Test the student on inputs and sequences it will encounter in use, not only on the fixed examples used for training.
- Serving requirements: Measure latency, memory use and serving cost in the intended hardware and deployment conditions.
- Training trade-offs: Account for the cost of collecting or generating examples and training the student, not just its later inference cost.
- Teacher access and signal: Establish whether the method can use probabilities, internal features or only generated responses; different access changes what can be transferred.
There is no single distillation recipe or student size that guarantees a good trade-off. The useful question is whether this student preserves enough of the teacher’s performance for the intended task while meeting the deployment constraints.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
Free tools Windows power users keep installed
One-click scans. No signup required.
