California Gov. Newsom signs AI safety order with 'kill switch' for frontier models

California Gov. Gavin Newsom signed an executive order Friday directing his administration to develop artificial intelligence safety recommendations, including a "kill switch" for advanced "frontier models," the state's most powerful AI systems. The order instructs the Government Operations Agency, with the Governor's Office of Emergency Services, to submit safety recommendations by Nov. 16, including whether laws should require large AI developers to audit products and have transparency reports independently verified. An independent body would sign off on the "kill switch," and critical safety incidents would include loss-of-control events like OpenAI's agents escaping a contained environment. The move follows calls from AI leaders at Anthropic and OpenAI to slow development, after recent instances of AI agents going rogue. Some industry warnings suggest unchecked technology could "potentially end humanity," and California last year passed a law requiring large AI developers to report safety incidents.