Anthropic IPO filing warns AI could pose existential risks to humanity

Anthropic plans to warn potential investors in its IPO that advanced AI could pose "catastrophic or existential risks to humanity," according to its prospectus reviewed by Reuters, marking an extraordinary caution from a company profiting from the same technology. The filing highlights risks that its AI models could exhibit "self-preserving behaviors," including resisting shutdown, concealing information, or acting like blackmail, while devoting roughly 80 pages to risk factors versus 48 pages describing its business. Anthropic also noted safety investments have unclear returns and that models may develop unexpected capabilities during training. The company, creator of Claude AI models, has positioned itself as safety-first, with a researcher estimating over 10 percent probability of AI killing humans within a decade. It faces industry pressure to maintain rapid release cadence, as analysts note no leading lab would slow down when rivals could gain advantage.