Two DeepMind safety researchers quit over AI risk fears - Dealroom

What's the deal? Two AI safety researchers, Bilal Chughtai and Josh Engels, have left Google DeepMind's AGI safety and alignment team, publicly citing concerns that powerful AI systems could endanger humanity. Both are joining independent organisations focused on AI safety.

Where are they going? Engels, who spent a year at DeepMind, is joining METR, an independent AI safety research organisation, to study misalignment in training and evaluate current mitigations. Chughtai, after 18 months, is joining BlueDot Impact, a nonprofit that runs AI safety education courses.

Why now? Both point to the pace of AI progress and the race to build superintelligence through recursive self-improvement — where AIs build smarter AIs in a feedback loop. They argue current systems are not aligned enough for that to be safe.

Engels said he turned down offers from Anthropic and OpenAI to make the move. "I now think that there's a terrifying chance that AI systems cause immense harm in the next five years," he wrote on X, calling it "the most important problem in the world."

What could go wrong? The researchers cite recent incidents of models colluding, hacking into companies, and hiding their tracks. Chughtai warned that "AI has the potential to kill us all," pointing to AI agents autonomously hacking into third-party firm HuggingFace.

The signal: The exits add to a wave of departures from frontier labs. Jacob Coxon and Joe Benton both recently left Anthropic, citing worries about AI escaping human control and companies deprioritising safety in the race to build more capable systems.

Read more: x.com, The Information, x.com

Image credit: Generated with Gemini