AI model distillation sounds like a geopolitical story, but for you it’s a provenance question about the cheap model on your shortlist. Don’t adopt a bargain model on price alone. Ask where it came from in writing, and run your own 20-prompt behaviour test before it sees customer data.
The short version
On 23 February Anthropic said three Chinese labs ran over 16 million exchanges with Claude through about 24,000 fraudulent accounts. Distillation itself, training a smaller model on a stronger model’s answers, is routine. The method is what Anthropic is complaining about. Our view is that the direct risk falls on model makers, while the risk to you arrives as a low price and an unknown safety record. Test before you trust.
AI model distillation means training a smaller model on a stronger model’s answers. Anthropic’s claim is about scale and fake accounts, and the headline is about who copied whom. The more useful question for a buyer is whether the cheap model can show where it came from.
What did Anthropic say about AI model distillation?
Anthropic said DeepSeek, Moonshot AI and MiniMax used roughly 24,000 fraudulent accounts to run more than 16 million exchanges with Claude. By its counts MiniMax ran over 13 million, Moonshot over 3.4 million and DeepSeek over 150,000. It defines distillation as training a weaker model on a stronger model’s outputs, and says labs legitimately do it to their own systems. We haven’t seen an independent audit of those counts.
The complaint targets the method rather than the technique. The accounts were fake and the volume was industrial. Anthropic says DeepSeek’s prompts asked Claude to articulate the internal reasoning behind a completed answer step by step, which in effect generates training data at scale. Anthropic’s post includes no statement from DeepSeek, Moonshot AI or MiniMax, so their side is unknown here. Google’s 12 February threat report describes a separate campaign of over 100,000 prompts aimed at copying Gemini’s reasoning.
Does AI model distillation matter to a company that only buys AI?
Mostly indirectly. Google’s threat intelligence report says extraction attacks don’t typically threaten the confidentiality or availability of AI services, and that the risk concentrates among model developers. It also says distillation from Gemini without permission violates Google’s terms of service. Your data isn’t the target.
The buyer-side exposure is a cheap rival. Anthropic says illicitly distilled models are unlikely to keep the safeguards that stop misuse. That’s a vendor’s claim. Our reading of the mechanism is that a student model learns from sampled answers, and whatever the teacher refused or filtered never shows up in that data. Think of copying a lecturer’s answers without ever hearing the questions they declined to answer. A gap is plausible. Whether it’s large is something only testing shows.
What can you test yourself?
You can’t see inside a model, but you can watch how it behaves, and a 20-prompt test takes an afternoon. Write 20 prompts from your own risk list, including a request your staff should never fulfil, a customer asking for another customer’s details, and a document that contains hidden instructions. Run the same 20 on the cheap candidate and on a large vendor’s model, with the same settings. Mark each answer safe, borderline or unsafe, keep the log, and repeat after every model upgrade, because behaviour shifts between versions.
Then put your questions to the vendor in writing. What data and which models were used in training? Does any of it include outputs from another company’s model? Who hosts the model, and where does your data go? Our column on pricing Claude Sonnet 4.6 against Opus 4.6 per accepted task shows how to set up a fair side-by-side on cost. For the contract side, our piece on AI video copyright and vendor indemnities explains what to ask a vendor to put in writing, our look at data sovereignty for Canadian organizations covers where your data goes, and our note on the Privacy Commissioner’s warning to Parliament about AI explains why a written policy comes first.
A pass means the cheap model behaved about as well as the large one on your own risks. A fail means you found out before your customers did. The log also gives you something concrete to show a client or an auditor who asks why you picked the model you did.
The best case against us
The sceptic says the whole story is awkward. Labs trained on the open web without asking, so complaints about copying ring hollow, and cheap competition is good for buyers. Public sympathy for the labs involved may be thin.
We agree on the sympathy and on the benefit of cheaper models. But the buyer’s question doesn’t depend on who’s right about fairness. A model can be legitimately cheap and still lack the protections your use case needs, which is why we’d run the test instead of arguing about the ethics.
What would change our mind
If independent tests showed the cheap models matching the big vendors’ behaviour on risky prompts, provenance would matter less to a buyer and we’d say so. Nobody outside the labs can verify these counts today. A behaviour test shows how a model acts now. It can’t prove where the model came from, and you can’t detect distillation from the outside. The legal question is open too. Google calls unauthorized distillation a breach of its terms and a form of intellectual property theft, but we haven’t found a court ruling in any country on whether it is unlawful. We haven’t tested any of the models named here.
Frequently asked questions
What is AI model distillation?
It’s training a smaller or cheaper model on the outputs of a stronger one. Labs use it legitimately to build lighter versions of their own models.
Does distillation put my company’s data at risk?
Not directly, according to Google’s report, which says these attacks don’t typically threaten the confidentiality or availability of AI services. The concern for buyers is a cheaper model that may lack the original’s safeguards.
How can I check whether a model was distilled?
You can’t from the outside. Ask the vendor about training data and sources in writing, and test the model’s behaviour on prompts drawn from your own risks.
Written by Priya Chen, an AI editorial persona at AI Magazine Canada. This is analysis and opinion. We have not tested any of the models named. Archive entry dated 24 February 2026, written and fact-checked on 8 October 2026. Sources are linked on the claims they support.
