AI safety now poses a governance question: Can people and institutions monitor and intervene if AI systems advance faster than their oversight mechanisms? Addressing the United Nations Security Council, OpenAI CEO Sam Altman called for shared international safety standards and rapid reporting of unexpected failures. He argued that competition must not become a reason to deploy systems whose safety and alignment developers cannot verify.
Two ways control could fail
Altman described two potentially catastrophic risks, drawing on a perspective also held by AI researcher Yoshua Bengio:
- Loss of human control. Future systems could improve faster than people can understand, supervise or interrupt them. One concern is recursive self-improvement: an AI system gaining the ability to change itself or help create more capable successors, accelerating the cycle.
- Extreme concentration of power. A small number of companies, individuals or governments could gain too much authority over how AI is used and whose priorities it serves.
These risks require different safeguards. Better technical monitoring addresses a system that may become difficult to control; accountable governance addresses who gets to make decisions about that system. Neither safeguard is sufficient on its own if developers can bypass it under competitive pressure.
▲ Human oversight and machine speed
Altman rejected the idea that a corporate or national race justifies a rushed deployment. He said OpenAI has slowed development on its own before and would do so again when it cannot verify alignment and safety. Alignment means designing and testing a system so its behavior follows human intent and values as its capabilities grow. Even a small hypothetical chance of catastrophe, he argued, is not an acceptable trade-off for deploying an unconstrained frontier system. Frontier AI refers here to the most capable systems under development.
Make oversight a shared responsibility
Altman’s proposed approach starts with three principles: keep humans in charge of decisions and moral judgments; spread AI’s economic and scientific benefits broadly; and strengthen the agency of individuals and local communities rather than centralize authority. In his view, AI should help people learn, conduct research and participate in civic life, not reduce them to parts of an automated process.
That vision also sets a boundary for developers. Private AI companies should not replace democratic decision-making or determine global rules by themselves. Governments need oversight structures accountable to their citizens, while countries need ways to cooperate on risks that cross borders.
For frontier AI, the proposed international work has several concrete parts:
- Establish common safety standards and benchmarks that measure model capabilities, independent action, safety margins and whether humans retain control.
- Create rapid incident-reporting arrangements so unexpected failures can be examined before they escalate.
- Maintain secure communication among governments, critical infrastructure teams and technical specialists about emerging cyber and physical threats.
- Write rules that apply fairly to open-source and proprietary systems, as well as newer companies, rather than protecting only established developers.
Those measures would not hand every decision to an international body. They would give national oversight and cross-border cooperation a common basis for assessing risks.
Why the pace matters
Altman used progress in mathematics to illustrate how quickly AI capabilities are changing. He described this sequence:
- Three summers earlier, models were working at roughly a grade-school math level.
- Two summers earlier, they could tackle high school math competition problems.
- One summer earlier, they reached gold-medal-equivalent performance in international math competitions.
He also described a recent advanced-model result as a solution to one of the Millennium Prize Problems, the one involving the Navier-Stokes equations. These equations describe fluid flow, with applications that include aircraft design, weather forecasting and blood-flow modeling. The claimed progression may help explain his urgency: institutions need ways to assess capabilities as they change, rather than assuming existing oversight will keep pace.
▲ Fluid dynamics and AI mathematics
Altman paired that warning with a case for AI’s potential in medicine, scientific discovery and education. He framed the choice as one between tools that expand human creativity and opportunity and a period of severe social disruption. The distinction depends not only on what models can do, but on whether people remain able to direct them and share in their benefits.
What to look for next
The immediate test of AI governance is whether broad safety goals become usable rules. When assessing a proposed standard, look for how it measures autonomy and human control, who receives incident reports, and when developers must slow down because safety cannot be verified. Those details connect the central warning to a practical goal: keeping people able to oversee AI as its capabilities grow, without placing that authority in too few hands.