jailbreak

The practice of bypassing an AI model's safety rules and restrictions with specially crafted prompts to elicit prohibited outputs.

2 articlesLast mentioned

In AI, a jailbreak is an attempt to bypass the safety rules and usage restrictions built into an AI model, such as a large language model, so that it produces outputs it was designed to refuse. Known techniques include role-play framing, hiding instructions in other languages or encodings, and gradually steering the model over a long conversation.

Model developers counter jailbreaks with safety training during model development and with classifiers that screen inputs and outputs; false positives from these classifiers, which block harmless requests, are a recurring point of contention about model restrictions. The related concept of prompt injection refers to manipulating a model through instructions hidden in external data, as opposed to a jailbreak, in which the user directly tries to remove the restrictions.

The term also refers to removing manufacturer restrictions on devices such as smartphones to install unofficial software.

This entry is based on AIPOST articles and widely known facts. If something is wrong, please send us a correction request.

Articles covering this entry

  • Claude Code Limits: Game Clone Ban, Cyber Filters and What $200 a Month BuysAI News ·

    …ense of what it calls a safety margin, openly stating that it tunes its classifiers to fire on prompts that are probably benign in order to keep jailbreaks out.

    Read article
  • Shipping a Customer-Facing AI Agent: Memory, Least Privilege and EvaluationAI in Practice ·

    It filters prompt injection, where an attacker slips in instructions to hijack the model, along with jailbreak attempts and leaks of sensitive data.

    Read article

Found a mistake or missing an entry?

Wiki entries are based on AIPOST articles and widely known facts. If an entry is wrong, send us a correction request and we will check and fix it. Names from new articles are added to the wiki after the article is published.