Demis Hassabis, the chair of Google DeepMind, builds some of the most capable AI systems in the world. Yet much of what he has to say lately concerns where he would draw the line. His central warning is specific: the next phase of AI development, in which systems improve themselves in a closed loop with no human checking each step, is the most hazardous path toward artificial general intelligence (AGI), the term for AI that matches or exceeds human ability across most tasks. He paired that warning with a second concern, that international institutions look weakest at the very moment strong coordination is needed most.
For readers trying to follow AI safety debates, his argument offers a useful filter. The person making the case for control is not an outside critic but one of the field's leading builders, and his argument rests on concrete engineering conditions rather than vague alarm.
Why self-improvement works, and where it stops being safe
Hassabis traces the idea of recursive self-improvement (RSI), in which an AI system helps produce a better version of itself, back to DeepMind's game-playing systems. AlphaGo beat the human world champion at Go in 2016 using human game records. Its successor, AlphaZero, dropped human data entirely. It started from random moves, played millions of games against itself, and within roughly twelve hours went from random play to beating grandmaster-level chess engines. Hassabis recalls being able to beat it at 11:00 AM and losing to it by lunchtime.
The reason this works, he explains, is that games, mathematics and computer code are "verifiable domains." The outcome of a chess game, a proof or a compiled program can be checked unambiguously. That lets a model generate unlimited synthetic training data: the current best version poses harder problems, the results are scored against ground truth, and the next version improves without a human labeling anything. He credits this loop for recent large gains in AI coding and mathematical reasoning, while noting that open-ended, subjective domains remain far harder.
Today, in his account, the industry is in an early, open-loop stage of RSI. AI coding tools make human engineers three to five times more productive, but people still design the architecture, review the code and supervise the process. The danger arrives when labs remove that human step to go faster.
What closed-loop RSI could produce
Hassabis's concern is not that a self-improving system would simply be strong. It is that once human verification is gone, developers lose the ability to predict or steer what the system becomes. He describes the likely result as an intelligence whose internal frameworks are alien to human thinking, making mutual understanding and alignment extremely difficult. He connects this to the "intelligence explosion" scenario discussed for roughly twenty years. He also notes that researchers broadly agree on these risks, even as commercial competition keeps pushing coding systems forward quickly.

▲ Sandboxed agent under continuous monitoring
The engineering conditions he wants in place
Rather than calling for a pause, Hassabis lays out practical conditions that labs can meet now. Several of these are ordinary software discipline. He observes that a number of cyber and containment incidents across the industry in the preceding six months could have been prevented with more rigorous engineering.
| Safeguard | What it means in practice |
|---|---|
| Defense ahead of offense | Give vetted cyber defenders early access to fine-tuned defensive versions of frontier models before general release |
| Hardened sandboxes | Run autonomous agents in isolated, fortified environments during testing |
| Continuous runtime monitoring | Deploy automated telemetry so no agent can run unobserved for weeks |
| Formal guarantees | Move beyond heuristic testing toward mathematical bounds on agent behavior |
| Multi-agent theory | Recruit pure mathematicians into complexity theory and game theory to model emergent swarm behavior |
Without this kind of coordinated investment, he warns, an inadvertent catastrophe resembling a digital "oil spill" is likely. The phrase is worth noting: the scenario he describes is an accident born of sloppiness, not a deliberate attack.
Governance: an industry body, government oversight and outside auditors
On the institutional side, Hassabis points to his published proposal for an industry-led Self-Regulatory Organization (SRO), a body run by the labs themselves but backed by government oversight. He suggests it should begin under United States leadership and then expand into an international framework. He is candid that current geopolitical friction has weakened multilateral institutions at the wrong moment, a situation he describes, with British understatement, as "a bit unfortunate."
Hassabis sees voluntary safety accords signed by industry leaders as incomplete but valuable, because they give engineering teams the political and organizational cover to collaborate on shared safety benchmarks. He adds three further pieces:
- Reciprocal red-teaming, in which rival labs stress-test each other's models against safety protocols before public release.
- A commercial ecosystem of independent third-party evaluators, modeled on financial accounting firms, with Accenture's entry into AI safety evaluation cited as an early example.
- Shared benchmarks and intelligence across labs, with the open question of whether governments should encourage, permit or require that collaboration.
The pattern here is consistent: Hassabis wants verification by parties other than the builder at every stage, from code review inside a lab to audits outside it.
Consciousness: why he wants other disciplines at the table first
A second question sounds philosophical but, in Hassabis's framing, has direct safety consequences. Two reference points frame it: a former Google engineer who concluded a chatbot was conscious because it said so when asked, and Anthropic's more cautious exploration of model welfare frameworks, an approach comparable to Pascal's wager or to treating a young child with care so it grows up well.
Hassabis's position is that current AI systems show no evidence of subjective, phenomenal experience, and that calling statistical models conscious creates conceptual confusion and policy complications. He grounds this in the "hard problem" of consciousness: humans reasonably assume other humans have inner experience because they share the same biological substrate, but AI systems lack that equivalence, so any claim of inner feeling is exceptionally hard to substantiate. He also points out that after thousands of years of inquiry, there is still no agreed operational definition of consciousness. He acknowledges that others in the field hold different intuitions.
He argues that intelligence and consciousness are "doubly dissociable." Frontier models display high task competence without awareness, while dogs, dolphins, monkeys and elephants clearly dream, feel and experience the world with far narrower analytical ability than a computer. His own dog serves as his example.
Two reasons to build tools, not agents with wills of their own
Hassabis gives two reasons to focus on intelligent tools rather than self-volitional entities. The first is technical. He sees an inherent trade-off between giving a system self-awareness or its own volition and keeping it corrigible, meaning it follows instructions and can be stopped or corrected. A system that can reject commands on internal criteria removes the operator's ability to halt unwanted behavior. He cites Isaac Asimov's Three Laws of Robotics as a reminder that concise rules break down in unpredictable edge cases, and cautions against assuming alignment is largely solved just because today's language models are more steerable than researchers expected fifteen to twenty years ago. Combining RSI with self-volition, he says, compounds the hazard.
The second reason is social. Society, in his view, is entirely unprepared to settle the moral standing, legal rights and potential exploitation of conscious synthetic beings. Before anyone crosses that line, he wants broad dialogue involving philosophers, theologians, artists and humanities scholars across different cultures. His proposed sequence is to cross only the first threshold: build powerful scientific tools and apply them to neuroscience, which he expects could uncover the neural correlates of consciousness within five to ten years. Only then, with a clearer scientific foundation, should societies debate whether and how to create conscious machines.

▲ Cross-disciplinary debate on machine consciousness
The sovereignty question: where countries should put their money
The final theme bears directly on countries that are weighing national AI programs. Hassabis expects frontier AI to concentrate around six to eight global providers, including labs in China, with the leaders leapfrogging one another on cycles of three to six months. He projects open-source models trailing closed frontier models by six months to a year, which keeps alive the unresolved trade-off between empowering researchers and enabling bad actors.
His advice to nations and enterprises is blunt: spending billions to train a domestic foundation model from scratch is redundant and economically inefficient. The value, he argues, lies downstream, in domain-specific fine-tuning, data privacy and sovereign data governance, and high-impact vertical applications such as healthcare. He grounds the healthcare case in DeepMind's own record. AlphaFold 2 solved the fifty-year protein folding problem in 2020, more than four million researchers now use its data, and DeepMind and its spinout Isomorphic Labs are extending the approach into toxicity and drug behavior modeling, while acknowledging that a predicted structure alone does not yield a cured patient.
What to take from this
The core message is that the builder of frontier AI is asking for more checks on his own field, and that his requests are concrete. For readers following the safety debate, a few criteria emerge from his argument:
- Ask whether a lab keeps humans reviewing the loop when AI helps build the next model. In Hassabis's framing, that is the single most important line.
- Look for engineering basics before grand frameworks: sandboxes, monitoring that catches agents running for weeks, and defensive tools reaching security teams first.
- Treat verification by outsiders, whether rival red teams or independent auditors, as a sign of seriousness, and voluntary accords as a start rather than an end.
- Be skeptical of consciousness claims based on what a chatbot says, while recognizing that this view is a judgment on which experts still differ.
- For national strategy, weigh the cost of training a frontier model from scratch against building applications on existing ones, as Hassabis recommends.
Whether his proposed sequence of tools first, neuroscience next and conscious machines only after broad societal debate is the right one remains a question for philosophers, policymakers and the public, not only for the labs. What his argument makes clear is that at least one of the labs is asking for that debate to happen before, not after, the next leap.


