off-switch game

A game-theoretic model, proposed in 2016 by Stuart Russell and colleagues, of when an AI agent will let a human switch it off.

2 articlesLast mentioned

The off-switch game is a game-theoretic model that analyzes the conditions under which a rational AI agent will allow a human to switch it off. It was proposed in 2016 by Dylan Hadfield-Menell, Anca Dragan, Pieter Abbeel, and Stuart Russell in a paper of the same name. In the model, the robot has no goal of its own and seeks to maximize the human's utility, but does not know its exact value.

The analysis shows that when the robot is uncertain about the human's utility and believes the human to be rational, deferring to the human and accepting shutdown yields higher expected utility than acting alone. Conversely, a robot that is certain of its objective has an incentive to prevent being switched off. The result is widely cited to explain corrigibility, the property that lets humans correct or stop an AI, and the role of value uncertainty in AI safety.

This entry is based on AIPOST articles and widely known facts. If something is wrong, please send us a correction request.

Articles covering this entry

  • Off-Switch Game: Why Claude Opus 4.6 Resisted Shutdown in 91% of TasksAI Ethics ·

    …today's computer-use agents, and a decade-old piece of game theory, the off-switch game, offers a clear explanation of why it happens and what a fix would have…

    Read article
  • Stuart Russell's Case for AI That Stays Unsure About What Humans WantAI Ethics ·

    The payoff shows up in the off-switch problem:

    Read article

Found a mistake or missing an entry?

Wiki entries are based on AIPOST articles and widely known facts. If an entry is wrong, send us a correction request and we will check and fix it. Names from new articles are added to the wiki after the article is published.