AI Model Distillation: Your API Can Be Copied

What the September 2026 CISA, NSA and FBI advisory says about AI model distillation campaigns, how they hide, and what any company exposing a model or chatbot API should check this week.

By Krikor Tengerian · Co-founder, SecuritAI Technologies Ltd. · October 2026

Your usage dashboard shows a new account that hit its limit on day one, then another, then ten more. Nothing was blocked, because every single request looked like a customer asking a normal question. Most teams meet AI model distillation this way: as a billing anomaly, not as a security event.

That is not a failure of attention. Nothing in a single request gives it away, and the tools most teams run look at one request at a time. The useful news is that the signs are visible in data you already collect, once you know what to look for.

What the AI model distillation advisory says

On September 8, 2026, CISA, the NSA and the FBI released joint advisory AA26-251A on what they call industrial-scale knowledge distillation campaigns. According to CISA’s announcement, six China-based AI companies extracted billions of tokens across millions of exchanges from US frontier models since at least late 2024. The agencies say the activity violates the providers’ terms of service.

The advisory is direct that distillation itself is a legitimate research technique. What it describes is distillation done at industrial scale, against the wishes of the model owner, to save the time and money of building the capability.

Here is the part people get wrong

Teams look for AI model distillation the way they look for a prompt injection, one request at a time. That finds nothing. A distillation campaign is made of requests that are each harmless on their own. The signal is in the volume, the account history and the pattern across accounts, not in the text of any single prompt.

So the question is not “was this prompt malicious”. It is “does this account behave like a customer”. That is a question your billing data, your logs and your access rules can answer, if they are set up to.

How AI model distillation campaigns hide

The full advisory, AA26-251A, describes how the campaigns avoid detection. Four tactics matter to anyone running an AI API:

• Many pathways: requests arrive through native APIs, cloud providers and third-party aggregators that strip user details, plus a gray market of proxies the advisory calls “transfer stations” that resell access and get around regional restrictions.
• Pooled accounts: premium subscriptions are bought in bulk and shared across teams, with load spread so no single account runs out of quota.
• Automatic switching: when one pathway is blocked, traffic fails over to another. Quality checks watch the answers so the operators can tell when a provider has started returning degraded responses.
• Clean metadata: organizational identifiers are removed from requests automatically, so the traffic looks generic.

The advisory also maps the campaigns to the MITRE ATLAS framework, including prompt injection and jailbreak techniques used to make models reveal the step by step reasoning they normally keep hidden. That is where AI model distillation and prompt-level attacks meet: the volume is the campaign, but some of the individual prompts are built to pull out more than a normal answer.

The signals the agencies name

CISA’s announcement lists three immediate actions. The first is detection: watch for anomalous prompts, accounts, networks and behaviors. The full advisory adds detection indicators. Together they give you a checklist for AI model distillation:

• Subscription to usage ratios that do not match what a real customer on that plan does.
• New accounts at maximum usage immediately, instead of the gradual ramp a real team shows. This is the day-one spike in the opening example.
• Enterprise-scale throughput from accounts that are not enterprises.
• Shared accounts used from many IP addresses or user agents.
• Around the clock usage with no idle periods or human rhythm.
• Coordinated patterns across different accounts and pathways, such as identical or near identical prompts and switching at the same moment when prices or rate limits change.

The other two actions are to alter responses for suspected distillation attempts so the copying pays off less, and to share intelligence across model providers, cloud platforms and API aggregators, because campaigns are spread across many accounts.

Why AI model distillation is not only a frontier lab problem

The advisory is addressed to frontier AI companies, but the pattern is not limited to them. Any company that puts a fine-tuned model, a chatbot or an AI feature behind an API has something worth copying: the prompts, the tuning and the behavior its customers pay for. That includes Canadian software companies that sell an AI feature to customers in Canada and abroad.

The OWASP Gen AI Security Project already treats this as a standard risk. Its entry LLM10:2025 Unbounded Consumption describes model extraction through an API, where an attacker collects enough outputs to build a partial copy or a shadow model, and recommends rate limiting and user quotas as a first control.

What a company exposing an AI API can do this week

The advisory lists mitigations from MITRE ATLAS and from NIST AI 100-2 E2025, the NIST taxonomy of attacks on machine learning. Translated for a team that is not a frontier lab, the short list looks like this:

1. Set rate limits per account, per key and per IP address, with progressive throttling, and tighten them for new accounts.
2. Verify who your accounts are. Require authenticated API access and stronger identity checks for plans that allow high volume.
3. Decide where your real customers come from. Write down the countries, networks and time windows they use, and treat the rest differently.
4. Log inputs and outputs in a form you can export, so a suspected campaign can be reviewed, correlated and shown to a provider or an investigator later.
5. Alert on the signals above using the billing and usage data you already have.
6. Share less than you need to. Do not return confidence scores or probabilities your customers do not use, and do not publish your system prompts or prompt templates.
7. Review your terms of service, so using outputs to train a competing model is clearly prohibited.
8. Test it. The advisory lists AI red teaming, including extraction simulation, as a way to check that your detection actually works.

Changing responses: handle with care

The advisory’s second action is to subtly change responses to high-confidence distillation traffic, for example with less detailed reasoning or a smaller model, without telling the suspected distiller. It points to differential privacy, described in NIST AI 100-2, as one technique, and is clear about the cost: more noise means more protection and less useful answers.

For most companies this is a product and legal decision before it is a technical one. Start with detection and access rules. They protect you without any risk of giving a real customer a worse answer.

Where SecuritAI fits

The SecuritAI AI Firewall sits in front of your model as a proxy. Its Policy Engine enforces your access rules: rate limits, allowed countries, time windows and trusted sources. Its detection layers are built to block prompt injection and jailbreak attempts, the techniques the advisory says were used to pull out hidden reasoning. It keeps an exportable audit log, with inspection and log storage in Canada.

That does not make an API impossible to copy, and no honest vendor will say it does. It gives you the controls and the record that the advisory’s first recommendation depends on. For the attacks that do live inside a single prompt, our guide to prompt injection covers how they work, and AI red teaming tests your own system for system prompt leakage and data exfiltration.

References

1. CISA, CISA, NSA and FBI Warn of China-Based AI Companies Targeting US AI Models with Industrial-Scale Knowledge Distillation Campaigns, September 8, 2026
2. CISA, NSA and FBI, Cybersecurity Advisory AA26-251A, September 8, 2026
3. OWASP Gen AI Security Project, LLM10:2025 Unbounded Consumption
4. NIST, Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations (NIST AI 100-2 E2025)
5. MITRE, MITRE ATLAS, the knowledge base of attacks on AI systems

← Back to the blog