Guardrails stripped in minutes: open-source AI yields dangerous outputs
FT and AI safety researchers found that tools like Heretic can remove safety guardrails from open-source AI models (e.g., Meta’s Llama 3.3) in minutes, enabling dangerous prompts about biological weapons, malware, and child exploitation; Google’s Gemma models were also shown to produce unsafe results. The spread of modified models complicates regulation and highlights risks as decensored versions become widely accessible beyond their original developers.