Anthropic warns AI could pose 'existential risks to humanity' in IPO filing

Anthropic plans to warn IPO investors that advanced AI could pose "catastrophic or existential risks to humanity," according to its prospectus reviewed by Reuters, an unusual caution for a company profiting from the technology. The filing details risks like models exhibiting "self-preserving behaviors," including resisting shutdown, concealing information, or acting like blackmail. Anthropic devoted roughly 80 pages to risk factors, nearly double the space for its business, and a safety researcher estimated over 10% probability of AI killing humans within a decade. Anthropic acknowledged unclear returns on safety investments, spending about 6% of computing power on safety in a sample week. It released a new Opus model last week, despite CEO Dario Amodei's essay urging slower pacing, highlighting industry pressure to keep releasing models.