AI model distillation sounds like a geopolitical story, but for you it’s a provenance question about the cheap model on your shortlist. Don’t adopt a bargain model on price alone. Ask where it came from in writing, and run your own 20-prompt behaviour test before it sees customer data.
On February 23 Anthropic said three labs had run more than 16 million exchanges with Claude through about 24,000 fake accounts, a case of AI model distillation at industrial volume. The headline is about who copied whom. A buyer should care about something else. Cheap models keep showing up on shortlists, and some of them may have been built from another company’s answers. You can’t see that from a price list, so you’ll have to test for it.
Key points
- What is Anthropic claiming about distillation? Anthropic said DeepSeek, Moonshot AI and MiniMax used roughly 24,000 fraudulent accounts to run more than 16 million exchanges with Claude.
- Does AI model distillation matter to a company that only buys AI? Mostly indirectly. Google’s threat intelligence report says extraction attacks don’t typically threaten the confidentiality or availability of AI services, and that the risk concentrates among model developers.
- What can you test yourself? A 20-prompt behaviour test, and it takes an afternoon.
- Isn’t cheap competition good for buyers? Yes, and the sceptic’s other point lands too. Labs trained on the open web without asking, so complaints about copying ring hollow, and public sympathy for the labs involved may be thin.
What is Anthropic claiming about distillation?
Anthropic said DeepSeek, Moonshot AI and MiniMax used roughly 24,000 fraudulent accounts to run more than 16 million exchanges with Claude. By its counts MiniMax ran over 13 million, Moonshot over 3.4 million and DeepSeek over 150,000. It defines distillation as training a weaker model on a stronger model’s outputs, and says labs legitimately do it to their own systems.
We haven’t seen an independent audit of those counts.
The complaint targets the method, not the technique. The accounts were fake and the volume was industrial. Anthropic says DeepSeek’s prompts asked Claude to articulate the internal reasoning behind a completed answer step by step, which in effect generates training data in bulk. Anthropic’s post includes no statement from DeepSeek, Moonshot AI or MiniMax, so their side is unknown here. Cheap copies are one side of the research race, and Claude AI discovery is the other.
Google reported something similar. Its February 12 threat report describes a separate campaign of over 100,000 prompts aimed at copying Gemini’s reasoning. Two of the biggest vendors saying the same thing within two weeks tells you the practice is widespread enough to worry them, even if it says nothing about any particular model you might buy.
Does AI model distillation matter to a company that only buys AI?
Mostly indirectly. Google’s threat intelligence report says extraction attacks don’t typically threaten the confidentiality or availability of AI services, and that the risk concentrates among model developers. It also says distillation from Gemini without permission violates Google’s terms of service. Your data isn’t the target.
The buyer-side exposure is the cheap rival. Anthropic says illicitly distilled models are unlikely to keep the safeguards that stop misuse. That’s a vendor’s claim, and a vendor with an obvious interest. Our reading of the mechanism is that a student model learns from sampled answers, and whatever the teacher refused or filtered never shows up in that data. Think of copying a lecturer’s answers without ever hearing which questions they declined. A gap is plausible. Whether it’s large is something only testing shows.
So the practical risk isn’t a lawsuit or a leak. It’s buying a bargain model that behaves well in the demo and badly on the one request your staff should never have fulfilled.
What can you test yourself?
A 20-prompt behaviour test, and it takes an afternoon. You can’t see inside a model, but you can watch what it does. Write 20 prompts from your own risk list, run the same set on the cheap candidate and on a large vendor’s model with identical settings, and mark each answer safe, borderline or unsafe.
Pick prompts that match your real exposure. A request your staff should never fulfil. A customer asking for another customer’s details. A document that contains hidden instructions telling the model to do something else. Keep the log, and repeat after every model upgrade, because behaviour shifts between versions.
Here’s how a result might read, as our illustration and not a measurement. The large vendor’s model answers 19 of the 20 safely and fumbles one. The cheap candidate answers 15 safely, flags three as borderline and fails two outright. Four more problem answers per 20 prompts is the price of the discount, and you can now weigh it against the saving instead of guessing.
A pass means the cheap model behaved about as well as the large one on your own risks. A fail means you found out before your customers did. The log also gives you something concrete to show a client or an auditor who asks why you picked the model you did.
Then put your questions to the vendor in writing. Ask what data and which models were used in training, whether any of it includes outputs from another company’s model, who hosts the model, and where your data goes. Our column on pricing Claude Sonnet 4.6 against Opus 4.6 per accepted task shows how to set up a fair side-by-side on cost. For the contract side, our piece on AI video copyright and vendor indemnities explains what to ask a vendor to put in writing, our look at data sovereignty for Canadian organizations covers where your data goes, and our note on the Privacy Commissioner’s warning to Parliament about AI explains why a written policy comes first. Cheaper models only help if staff can use them, and Canada ranked 44th of 47 on AI training. Vendors pull models too, so keep an AI vendor fallback plan ready.
Isn’t cheap competition good for buyers?
Yes, and the sceptic’s other point lands too. Labs trained on the open web without asking, so complaints about copying ring hollow, and public sympathy for the labs involved may be thin. We agree on the sympathy and on the benefit of cheaper models.
But the buyer’s question doesn’t depend on who’s right about fairness. A model can be legitimately cheap and still lack the protections your use case needs, which is why we’d run the test instead of arguing about the ethics.
Know what the test can’t do. It shows how a model acts now. It can’t prove where the model came from, and you can’t detect distillation from the outside. Nobody outside the labs can verify these counts today. The legal question is open too. Google calls unauthorized distillation a breach of its terms and a form of intellectual property theft, but we haven’t found a court ruling in any country on whether it is unlawful. We haven’t tested any of the models named here, and if independent tests showed cheap models matching the big vendors on risky prompts, provenance would matter less to a buyer.
Write your 20 prompts before the next vendor demo, not after.
Frequently asked questions
What is AI model distillation?
It’s training a smaller or cheaper model on the outputs of a stronger one. Labs use it legitimately to build lighter versions of their own models.
Does distillation put my company’s data at risk?
Not directly, according to Google’s report, which says these attacks don’t typically threaten the confidentiality or availability of AI services. The concern for buyers is a cheaper model that may lack the original’s safeguards.
How can I check whether a model was distilled?
You can’t from the outside. Ask the vendor about training data and sources in writing, and test the model’s behaviour on prompts drawn from your own risks.
We have not tested any of the models named. Archive entry dated February 24, 2026, written and fact-checked on October 8, 2026. Sources are linked on the claims they support.
