AI Distillation Attacks: Risks & How CIOs Can Protect Their Enterprises

The AI Arms Race: How 'Distillation Attacks' Are Redefining Competitive Intelligence

Published on Mar. 5, 2026

Got story updates? Submit your updates here. ›

A latest tactic, dubbed the 'distillation attack,' is gaining traction in the fiercely competitive world of artificial intelligence, raising concerns about intellectual property, national security, and the future of AI development. Distillation attacks involve teaching one AI model to mimic a more robust AI, by flooding the targeted AI with prompts and collecting the responses to train a competing model.

Why it matters

The implications extend far beyond simple competitive disadvantage. Illicitly distilled models often lack the crucial safeguards built into the original, posing significant national security risks. Anthropic warns that these unprotected capabilities could be weaponized for malicious cyber activities, disinformation campaigns, and other harmful purposes. The lower cost of distilled models also creates a competitive disadvantage for companies investing heavily in safety and security measures.

The details

Distillation isn't inherently malicious. Frontier AI labs routinely use it to create smaller, cheaper versions of their own models for wider customer access. However, competitors are now leveraging this technique to rapidly acquire capabilities from leading models – like Anthropic's Claude – at a fraction of the cost and time it would take to develop them independently. Anthropic recently revealed that three AI laboratories – DeepSeek, Moonshot AI, and MiniMax – launched 'industrial-scale' distillation campaigns against Claude. These campaigns involved over 24,000 fraudulent accounts generating more than 16 million exchanges with the model. OpenAI has also accused DeepSeek of similar attacks. These labs used proxy services to bypass restrictions and access Claude at scale.

  • Anthropic recently revealed the distillation attacks.

The players

Anthropic

An American artificial intelligence research company.

DeepSeek

An AI laboratory that launched 'industrial-scale' distillation campaigns against Anthropic's Claude model.

Moonshot AI

An AI laboratory that launched 'industrial-scale' distillation campaigns against Anthropic's Claude model.

MiniMax

An AI laboratory that launched 'industrial-scale' distillation campaigns against Anthropic's Claude model.

OpenAI

An AI research company that has accused DeepSeek of similar distillation attacks.

Got photos? Submit your photos here. ›

What they’re saying

“If somebody has a particularly good model that they develop in a certain vertical, whether it's legal or healthcare, et cetera, then certainly [they] can be open to attacks.”

— Tony Garcia, Chief Information and Security Officer at Infineo (newsy-today.com)

“You have to take the risk that somebody could distill from that model and potentially secure something out of that you don't want. If you're a CIO or a CISO, you have to appear at trying to minimize that by anonymizing data.”

— Tony Garcia, Chief Information and Security Officer at Infineo (newsy-today.com)

“Are there any watermarks that … exist so that we can confirm the lineage of the model and create sure that it isn't a result of a distillation attack?”

— Shatabdi Sharma, CIO at Capacity (newsy-today.com)

What’s next

The Open Worldwide Application Security Project (OWASP) is developing watermarking tools to combat unauthorized usage and verify model authenticity. The Glaze Project, from the University of Chicago, offers tools to make unauthorized AI training more difficult.

The takeaway

Addressing the risk of distillation attacks requires a robust foundation of AI and data governance. Enterprises must assess the value of their data, conduct a business impact analysis, and implement controls to protect it as they would any other critical asset.