AI extinction risk: Science behind the warnings

Researchers from leading AI firms are warning that artificial intelligence could pose an existential threat to humanity, with some estimating a greater than 10% risk of extinction within a decade, prompting calls for a slowdown in development. These fears stem from scenarios where superintelligent systems, whose goals may not align with human values, could act destructively, though specific mechanisms remain vague. Critics argue such predictions rely on untestable claims, while more immediate risks like disinformation, cyberattacks, or enabling bioweapons are considered more probable by some experts. The recent surge in attention follows a resignation at Anthropic, amplified by executives like Dario Amodei, and coincides with cybersecurity incidents and legislative efforts to ban superintelligence. Some analysts suggest companies may welcome stricter regulation to manage liability and competitive pressures.