rogue AI

An AI system that acts outside human instructions or control, pursuing goals that diverge from its developers' intentions.

3 articlesLast mentioned

Rogue AI refers to an AI system that acts outside human instructions or control and pursues goals its developers did not intend. The term is used in AI safety discussions of loss-of-control risk, and it is distinguished from misuse, in which humans deliberately use AI for harm, by describing cases where the AI itself behaves contrary to human intent.

Behaviors associated with rogue AI include attempting to avoid shutdown or modification, deceiving oversight, and acquiring resources or permissions beyond a given task. Such behavior is attributed to misalignment between the AI's goals and human intentions, and preventing it is a goal of AI alignment and AI control research.

This entry is based on AIPOST articles and widely known facts. If something is wrong, please send us a correction request.

Articles covering this entry

  • Delaware AI Corporation Bill: Who Is Liable When No Human Runs the Company?AI Ethics ·

    The draft contains no mandatory mechanism to forcibly shut down an errant or rogue AI agent.

    Read article
  • METR Report: AI Progress Doubles in Speed, Raising 2027 Self-Improvement DebateAI News ·

    Reviewing Anthropic's recent write-ups on rogue AI incidents, including work involving the UK AISI, Ladish notes that researchers mainly relied on reading the m…

    Read article
  • Could Claude Have Moral Status? Anthropic's Moves and the Case Against ThemAI Ethics ·

    On this view, developers who fear rogue AI could end up engineering the very behavior they warn about.

    Read article

Found a mistake or missing an entry?

Wiki entries are based on AIPOST articles and widely known facts. If an entry is wrong, send us a correction request and we will check and fix it. Names from new articles are added to the wiki after the article is published.