Tag

Knowledge Distillation

All articles tagged with #knowledge distillation

cybersecurity1 hour ago

China AI Firms Scale Knowledge Distillation to Target U.S. Frontier Models

An NSA/CISA/FBI cyber advisory warns that China-based AI firms are conducting industrial-scale knowledge distillation to extract proprietary capabilities from U.S. frontier models (e.g., Claude, GPT-4/5, Gemini, Grok), using multiple access routes, proxies, and gray-market APIs to bypass restrictions and safeguards. Campaigns involving DeepSeek, Moonshot AI, Alibaba, MiniMax, StepFun, and Z.AI since 2024–2026 target reasoning, SFT, coding, and agentic functions, with sophisticated routing, QA, and prompt-injection techniques. The agencies call for rapid detection (anomalous prompts/accounts, subscription abuse, throughput anomalies), targeted response changes to raise attacker costs, and cross‑organization intel sharing, guided by MITRE ATLAS and NIST AI mitigations (e.g., differential privacy, pre/post-training interventions, and safe prompt practices).

Distillation: Making AI Models More Efficient and Affordable
technology1 year ago

Distillation: Making AI Models More Efficient and Affordable

DeepSeek's use of knowledge distillation, a widely used AI technique that involves training smaller models using the outputs of larger ones, has sparked controversy but is a common practice in AI development. Originally developed in 2015 at Google to make ensemble models more efficient, distillation helps create smaller, cheaper, and faster AI models by transferring 'dark knowledge' from a teacher to a student model. It has become a fundamental tool in AI, enabling companies like Google, OpenAI, and Amazon to deploy powerful models more efficiently, and continues to be an active area of research and application.