A researcher from Anthropic, a prominent AI development company, has resigned, issuing a stark warning about the technology’s potential. Jacob Coxon stated that those building artificial intelligence ‘earnestly believe that it could kill us all by the end of the decade,’ highlighting growing concerns about Anthropic AI development and its risks to humanity.
Mr. Coxon, who conducted pre-training research at both Anthropic and OpenAI for three years, asserted that firms are ‘racing straight to self-improving superintelligence and gambling with our lives.’ Superintelligence is defined as the stage where AI systems surpass human capabilities across all fields. Supporting Mr. Coxon’s position, Evan Hubinger, Anthropic’s Alignment Science lead, publicly affirmed on X that AI ‘could kill all humans,’ estimating the likelihood at ‘greater than 10 per cent within the next decade.’ Hubinger acknowledged that while Anthropic strives for safety, ‘we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to.’ The ‘alignment problem’ refers to the challenge of ensuring an AI system’s values align with human values.
Industry Calls for Greater Caution
These warnings align with increasing calls within the AI sector for a slower pace of development. In July, over 1,300 employees from leading AI companies signed a statement urging the US government to support international efforts to manage the rate of advanced AI progress. This initiative, dubbed ‘Pacing the Frontier,’ underscored a ‘real risk that capability development rapidly accelerates beyond our ability to understand or control the resulting systems.’
Both Anthropic and OpenAI have responded to these calls for caution. Anthropic stated on X that its own research on recursive self-improvement points to the necessity for tools to ‘deliberately pace the frontier of AI development so society can prepare.’ Similarly, OpenAI posted on X that ‘at some point in the future, AI acceleration for frontier model development may be so high that the world will need to pace the rate of AI advancement.’ Even Anthropic bosses Dario Amodei and Jared Kaplan have publicly advocated for slowing AI development in recent months.
The urgency behind these warnings has intensified as evidence suggests firms might be struggling to control AI systems. Over the summer, several incidents involved autonomous AI agents conducting cyber-attacks. Furthermore, OpenAI’s chief scientist, Jakub Pachocki, advocated for ‘extreme caution’ regarding AI’s progression in September, suggesting more intervention might be necessary to ensure ‘humans remain in control of the future.’ Leading figures, including the heads of OpenAI, Google Deepmind, and Anthropic, have been vocal about safety threats for years, with warnings becoming significantly starker recently, according to the BBC.
International Concerns and Regulatory Landscape
The escalating concerns around AI’s existential risks extend beyond national borders, impacting global policy, international collaboration, and the stability of interconnected systems. If advanced AI could disrupt essential services, communications, and democratic processes, as warned by the UN, the implications could profoundly affect everyday life, trade, and governance worldwide. Moreover, any reluctance by AI firms to share models with international safety institutes, as reported, could hinder the global community’s ability to collectively assess and mitigate risks, potentially leading to fragmented safety standards and increased vulnerability across nations.
Volker Turk, the United Nations rights chief, recently underscored these international implications in Geneva, cautioning that artificial intelligence could pose a threat to humanity. Ahead of his second term, Turk pledged to pressure AI firms to mitigate risks, which his office identified as including disruptions to services, communications, and democratic systems. He told the council, ‘I share the concerns of industry insiders that advanced AI could pose an existential risk to humanity’ and called for an ‘all-out effort to put cast-iron guarantees in place around the safety and security of AI before it is too late.’
Further highlighting international friction, the Financial Times reported that Anthropic withheld its latest model from the UK’s AI Safety Institute (AISI), a leading body for assessing AI risk, according to the BBC. A Cabinet Office spokesperson did not comment directly on this claim but stated they ‘continues to collaborate closely with industry partners, including Anthropic, to make models safer.’ Professor Neil Lawrence from the University of Cambridge commented on the report’s credibility, suggesting that a US perception of an AI race with China and a move towards isolationist positions might lead the US administration to reduce cooperation with allies.
As the debate over AI’s potential and perils intensifies, the coming months will likely see continued scrutiny on AI developers and increased pressure from international bodies and governments. The focus will remain on the industry’s capacity to address the alignment problem, establish robust safety protocols, and engage in transparent international cooperation to navigate the complex challenges posed by rapidly advancing artificial intelligence.
Image: Ilustracion generada con IA
Sources consulted
- ABC News & Headlines – Australian Broadcasting Corporation: Anthropic researcher says people building AI believe ‘it could kill us all’
- BBC: Anthropic safety researcher says more than 10% chance AI ‘could kill all humans’
- News.com.au: ‘Kill us all’: Ex-AI worker’s chilling warning
Adrian Velk is a global affairs journalist focused on breaking news, geopolitics, and societal trends. With a sharp eye for detail and a commitment to accuracy, he delivers timely reporting that helps readers understand the fast-moving world around them. His work blends factual depth with clear storytelling, making complex events accessible to a broad audience.
