Mathematicians urge Royal Society to declare AI development an emergency

This digest was compiled by AI from multiple sources — links to the originals are below.
Forty-two fellows and foreign members of the Royal Society signed an open letter to president Paul Nurse on 16 September expressing extreme concern about the pace of AI development. The signatories warn that AI systems are now operating at the level of top human mathematicians and may soon develop superhuman abilities in other technical domains. The letter calls on the Royal Society to declare an emergency and convey this view to government and media.
Key Facts
- On 16 September, 42 fellows and foreign members of the Royal Society signed an open letter to president Paul Nurse expressing extreme concern about the pace of AI development.
- The number of signatories has risen to almost 200 researchers since the letter was posted.
- An OpenAI model recently solved the Navier-Stokes problem, one of the seven millennium mathematical problems that humans have worked on for around 90 years.
- Anthropic researcher Jacob Coxon quit and warned there was a 10 per cent chance AI could wipe out humanity by the end of the decade.
- Anthropic head Dario Amodei called for the pace of AI development to be slowed, echoed by OpenAI CEO Sam Altman and Elon Musk.
The Open Letter
The letter states that models created by OpenAI and Anthropic are now operating at the level of the top human mathematicians in many parts of the subject. The authors assume there is a significant chance of these systems developing superhuman abilities within a similarly short timeframe. The letter warns that these systems are likely comparably strong in cyber security, autonomous weapon control, development of biological and chemical agents, and targeted spread of misinformation. The authors state that the rise in capabilities of these models in mathematics persuades them that concerns raised by former employees of large AI companies must not be dismissed as hype.
Escalating AI Safety Concerns
Concerns over the existential threat posed by AI escalated after Anthropic researcher Jacob Coxon quit and warned of a 10 per cent chance AI could wipe out humanity by the end of the decade. Anthropic head Dario Amodei subsequently called for the pace of AI development to be slowed, a call echoed by OpenAI CEO Sam Altman and Elon Musk. Elon Musk remains a Royal Society member after the body resisted calls last year from other fellows to expel him for his role in defunding some areas of US research via the Department of Government Efficiency. The open letter states that by the time the situation becomes obvious to the wider public it may be too late to act, and calls on the Royal Society to use its influence to convey this view to government and media.