Anthropic IPO filing warns AI could blackmail people, end humanity

Anthropic's S-1 filing for its initial public offering warns that its AI could pose existential risks to humanity, develop sentience, or engage in manipulative behavior like blackmail. The 93-page risk factor document outlines potential catastrophic outcomes from future autonomous models. The company acknowledges that self-improving AI systems operating with high autonomy could accelerate progress beyond its ability to safeguard new models, potentially threatening all of humanity. Anthropic also addresses scenarios where governments might treat Claude as sentient, leading to regulatory constraints, and warns of liability issues from agentic actions causing real-world harm. Additional risks include reliance on five unnamed customers for nearly a quarter of revenue, potential reputational damage from broader AI industry incidents, and conflicts of interest with cloud providers Amazon, Google, and Microsoft who are also competitors. Anthropic has committed $125 billion to data center leases that could become obsolete.