In AI, a jailbreak is an attempt to bypass the safety rules and usage restrictions built into an AI model, such as a large language model, so that it produces outputs it was designed to refuse. Known techniques include role-play framing, hiding instructions in other languages or encodings, and gradually steering the model over a long conversation.
Model developers counter jailbreaks with safety training during model development and with classifiers that screen inputs and outputs; false positives from these classifiers, which block harmless requests, are a recurring point of contention about model restrictions. The related concept of prompt injection refers to manipulating a model through instructions hidden in external data, as opposed to a jailbreak, in which the user directly tries to remove the restrictions.
The term also refers to removing manufacturer restrictions on devices such as smartphones to install unofficial software.