An AI distillation attack uses repeated access to a model API to collect its outputs and train an unauthorized “student” model to reproduce selected capabilities. Distillation itself is a legitimate machine-learning technique; the security issue is covert extraction without permission. For API operators, the practical response is layered: spot suspicious patterns across accounts, control access and output exposure, and investigate before restricting legitimate users.
How a distillation attack works
A model API returns answers to user queries. An extractor can automate prompts aimed at a valuable capability, collect the resulting answers as training examples, then use supervised fine-tuning or another training method to transfer some of the teacher model’s behavior to a student. Google Threat Intelligence Group describes this broader activity as model extraction and notes that legitimate API access can be used to try to clone selected capabilities, including reasoning, coding, data analysis, and tool use: GTIG AI Threat Tracker.
As an Amazon Associate I earn from qualifying purchases.
The distinction is authorization and context, not the mere use of distillation. Knowledge distillation is a common technique for training student models from teacher models, and it has legitimate applications. Ordinary experimentation, evaluation, or an authorized training arrangement should not be treated as malicious without evidence.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
A single prompt is rarely diagnostic. Requests that look harmless individually may reveal a different picture when they recur at scale, follow similar templates, concentrate on a narrow capability, or appear coordinated across accounts and intermediaries.
#1 Best Overall
What suspicious activity can look like
Look for combinations of signals over time, rather than a single prompt or fixed request-rate threshold. Anthropic’s February 2026 account of investigated campaigns describes high volume focused on a few areas, repetitive structures, and prompts aligned with valuable training capabilities as hallmarks. Those patterns are grounds for review, not proof of intent.
- Request or output volume is unusually high for the account’s stated use or established baseline.
- Prompts repeatedly use the same structure, with small variations that could generate many related examples.
- Traffic focuses disproportionately on a valuable capability, such as reasoning, coding, agentic tool use, or data analysis.
- Timing, infrastructure indicators, or similar request patterns link activity across accounts.
- Requests seek hidden reasoning or detailed internal traces that are not part of the API’s intended output.
- Repeated account creation, suspicious verification patterns, or proxy-mediated access accompany the traffic.
Batch inference, evaluation, research, and enterprise workloads can produce some of the same signals. Combine activity with account context and changes over time, then use a risk score or human review rather than automatically blocking users for one prompt pattern. The cited sources do not establish universal numeric thresholds.
Rank #2
What reported campaigns show—and do not show
In a February 23, 2026 disclosure, Anthropic reported more than 16 million exchanges across approximately 24,000 fraudulent accounts in three campaigns it attributed to DeepSeek, Moonshot, and MiniMax. It also described a proxy network managing more than 20,000 fraudulent accounts simultaneously and mixing distillation traffic with unrelated customer requests. These are Anthropic’s figures for campaigns it investigated, not an independently measured industry-wide rate.
The Tool Desk
Outbyte PC Repair FREEClear out junk files and repair common Windows errorsFree Scan →Outbyte Driver Updater FREEScan for outdated or missing drivers - takes under a minuteDriver Scan →The same disclosure attributed more than 13 million exchanges to the MiniMax campaign and more than 150,000 to the DeepSeek campaign. Anthropic said the DeepSeek activity targeted reasoning, rubric-based grading, and policy-sensitive query alternatives. The examples illustrate why account-by-account monitoring alone may miss coordinated activity; they do not establish a typical campaign size or prove that any given high-volume customer is extracting a model.
Rank #3
A layered protection plan for a model API
1. Set access controls around expected use
Protect API keys and apply quotas at the account and project levels. Verify accounts in proportion to the sensitivity and scale of the service, and review elevated-access routes such as research or education programs. Anthropic says it strengthened verification for several account types it considered vulnerable to fraud. These measures can reduce access or limit the volume available to an account, but individual limits may not reveal traffic distributed across many accounts.
2. Detect repeated behavior and correlate accounts
Use rules or classifiers to flag repeated prompt structures, unusually high volume, and sustained focus on particular capabilities. Where policy and law permit, correlate activity across accounts and projects; proxy networks or coordinated account creation can make each account appear less unusual in isolation. Anthropic describes using behavioral fingerprinting and detecting coordination across accounts.
Rank #4
- API Security in Action
- Manning Publications
- ABIS BOOK
Set investigation thresholds against your own legitimate workloads instead of importing a universal request limit: none is established by the cited sources. Review changes from an account’s baseline and the combination of signals, and give security or operations staff a path to assess ambiguous cases.
Quick wins for a faster PC:
Clear out junk files and repair common Windows errorsFree Scan →Scan for outdated or missing drivers - takes under a minuteDriver Scan →3. Limit exposure without breaking the service
Use quotas, rate limits, and output controls proportionate to expected use and observed risk. Consider whether every endpoint needs the same level of access or output detail. A graduated response can move from additional verification to throttling, review, or suspension as the evidence strengthens. No specific rate-limit configuration is prescribed by the cited sources; the right settings depend on the API’s architecture and customer workloads.
Best Value
4. Keep unintended internal details out of responses
Return what the user-facing task needs, not internal traces or implementation details that are not intended as API output. Google GTIG reports attempts to coerce reasoning traces from a model and notes that internal traces are typically summarized before delivery to users. This is an output-design measure, not a guarantee that model behavior cannot be learned from ordinary answers.
5. Use watermarking as a possible signal, not a shield
Watermarking may help with traceability, but it should not be treated as a standalone extraction barrier. A 2025 ACL paper by Pan and colleagues studied two teacher–student model pairs and two watermark schemes in its primary experiments. It reports that targeted paraphrasing and inference-time watermark neutralization removed inherited watermark signals while retaining distilled knowledge in the tested settings. That result demonstrates a limitation in those experiments; it does not establish that every watermarking method fails in every deployment.
6. Coordinate the response
When appropriate, share technical indicators with trusted providers and relevant authorities. Review detections with security, product, legal, and customer teams so that containment is proportionate and legitimate workloads are considered. Anthropic describes intelligence sharing as part of its response.
Do these 3 things before closing this tab:
1Clear out junk files and repair common Windows errors2Scan for outdated or missing drivers - takes under a minute3Repair Windows errors before they cause bigger problemsHow to weigh the trade-offs
| Control | Primary role | Trade-off or limitation |
|---|---|---|
| Account verification and quotas | Reduce unauthorized access or limit volume per account. | May add friction for legitimate customers; per-account limits can be evaded by distributing traffic. |
| Behavioral detection and cross-account correlation | Identify repeated prompts, capability concentration, and coordination. | Requires context and review because legitimate batch jobs, evaluation, and research can resemble extraction. |
| Output controls | Reduce unnecessary detail or access to sensitive output. | Can reduce usefulness if controls suppress information the intended task requires. |
| Watermarking | Potentially support traceability of outputs or downstream models. | Not a prevention guarantee; tested removal methods were effective in the settings examined by Pan et al. |
| Indicator sharing | Help relevant organizations recognize coordinated activity. | Requires suitable partners and careful handling of legal, policy, and customer considerations. |
What older extraction research can—and cannot—tell operators
Krishna and colleagues’ 2020 study, “Thieves of Sesame Street: Model Extraction on BERT-based APIs”, reported a query budget of less than $400 in its BERT-based API extraction setting. The authors also described full extraction as an open problem despite tested defenses. This is a historical, task-specific result—not a current cost estimate for extracting a frontier large language model.
Neither that study nor the more recent organizational disclosures provide universal account-count limits, request-rate thresholds, or a configuration recipe that suits every API. Choose controls according to the service’s exposure, expected customer behavior, and ability to correlate activity, then reassess them as usage patterns change.
Quick Recap
Sources
- Anthropic, “Detecting and preventing distillation attacks,” February 23, 2026
- Google Cloud / Google Threat Intelligence Group, “GTIG AI Threat Tracker: Distillation, Experimentation, and (Continued) Integration of AI for Adversarial Use,” February 2026
- Krishna et al., “Thieves of Sesame Street: Model Extraction on BERT-based APIs,” ICLR 2020
- Pan et al., “Can LLM Watermarks Robustly Prevent Unauthorized Knowledge Distillation?,” ACL 2025
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
