Chinese AI labs created 24K accounts and «distilled» 16 million messages from Claude

Chinese attacks risk bypassing the safeguards Anthropic builds into its models. (Picture: Anthropic)
Anthropic claims to have discovered industrial scale extraction of Claude data from DeepSeek, Moonshot AI and MiniMax.

The massive attacks were used to improve their own models with agentic reasoning, tool use, and coding capabilities, violating Anthropic’s Terms of Service and creating a national security risk, they say.

Distillation works by sending millions of prompts to an AI to incorporate its techniques and capabilities into their own models, drastically reducing training time and costs.

They also circumvent Anthropic’s protections for use in developing bioweapons and malicious cyber activities, Anthropic says. Once these models are open sourced, this becomes available to anyone.

OpenAI said the same just last week, accusing DeepSeek of distillation.

— These campaigns are growing in intensity and sophistication. The window to act is narrow, and the threat extends beyond any single company or region, Anthropic writes.

Read more: Anthropic’s announcement, writeups on Reuters, TechCrunch, Engadget and The Verge.

OpenAI sees rise in China-based ChatGPT abuse

OpenAI reports on Cyber Threats using ChatGPT.
We should probably be more worried about what ChatGPT doesn’t catch. (Picture: howtostartablogonline.net, CC BY 2.0)
China and Iran are using ChatGPT for influence operations, while North Korea and Russia looks for jobs backdoors and malicious code.

Out of the ten campaigns identified in OpenAIs new report «Disrupting Malicious Uses of AI», four were from China.

Supercharging influence ops
Chinese groups have used ChatGPT for mostly adversarial influence operations, writes Reuters, generating social media posts on political tops including on a Taiwanese video game, accusations against a Pakistani activist and content related to the closure of USAID.

Continue reading “OpenAI sees rise in China-based ChatGPT abuse”