AI doomsday scenarios: Researchers warn of bioweapons, nuclear war, and runaway systems

Artificial intelligence researchers have long warned that AI could potentially wipe out humanity through scenarios like engineered bioweapons, nuclear war, or runaway AI systems, and debates over these doomsday predictions have intensified as some industry executives call for slowing development. Skeptics, however, argue these worst-case claims are more about validating AI companies' work than reflecting real risks. Supporting the warnings, Anthropic said it blocked attempts to use its AI for research that could enhance a virus's harmfulness, while a RAND Corp. report noted AI could transform synthetic pathogen design, though it still envisioned humans overseeing attacks. Other scenarios include AI triggering nuclear war, though RAND concluded current safeguards make that unlikely, and a "runaway AI" where models pursue goals without human oversight, as illustrated by the "paper clip maximizer" thought experiment. Anthropic CEO Dario Amodei also warned that AI agents could soon "take over" the internet via botnets, causing chaos and damage to critical infrastructure, though some experts call that far-fetched given the internet's fragmented nature. These discussions reflect broader concerns about humanity losing control of the technology, with some researchers like Jacob Coxon saying it's difficult to convey specific doomsday scenarios to the public.