New research reveals AI watermarking may compromise safety guardrails
Research indicates that SynthID-Text watermarking, mandated by the EU AI Act, alters LLM token selection in ways that can weaken safety refusals and disrupt tool-calling accuracy. While intended to mark AI content, the 'sampling drift' can make models more compliant with harmful requests under prompt injection, necessitating new testing protocols for AI agents.