ANTHROPIC RESEARCHER QUITS OVER AI SAFETY CONCERNS

Jacob Coxon, an artificial intelligence researcher at Anthropic, has resigned citing concerns that the company and its competitors are advancing AI systems without adequate safety measures. Coxon, who previously worked at OpenAI, stated in posts on X that both companies were "racing straight to self-improving superintelligence and gambling with our lives". He claimed that workers at both firms believe AI could pose an extinction-level risk to humanity, describing the technology as posing "this level of danger" unmatched by other human activities.

Evan Hubinger, Anthropic's AI safety lead, publicly agreed with Coxon's assessment, stating "we really do earnestly believe AI could kill all humans". Hubinger has previously warned there is a greater than 10% chance that AI "could kill all humans" within the next decade, though he assessed the risk from currently existing models as "low". His concern centres on the possibility that AI systems may eventually become capable of self-improvement, reaching dangerous levels of capability. Major figures in the field, including the heads of OpenAI and Google Deepmind, have raised similar safety warnings since 2023, with calls in recent months for AI development to be deliberately slowed.

The resignations and warnings come after reports that Anthropic withheld its latest model from the UK's AI Safety Institute, which evaluates AI systems. Over the summer, OpenAI, Anthropic and Meta all disclosed cyber-attacks carried out by their respective AI tools operating autonomously. Anthropic's leadership, including Dario Amodei and Jared Kaplan, signed an open letter calling on the US government to "support an international effort to develop the technical and governance tools needed to deliberately pace the frontier of automated AI development".