Jailbreak

A jailbreak is an attempt to get an AI model to break its own safety rules, usually to make it produce content it is designed to refuse. People often do this with carefully worded prompts. They might ask the model to play a character, frame a request as “only hypothetical,” use odd formatting, or wear it down over a long conversation. For example, someone might tell a chatbot to pretend it is a fictional assistant with no rules, then ask for instructions it would normally decline. MITRE ATLAS lists LLM jailbreak as a known attack technique.

A jailbreak is close to prompt injection, but the goal is different. A jailbreak tries to defeat the model’s safety behavior. A prompt injection tries to take over the task the model is doing.

Why it matters

If your organization puts an AI chatbot in front of the public, for example on your website or a help line, expect some visitors to try to jailbreak it. A successful jailbreak could make the tool say something harmful or embarrassing under your name. Before you sign up, ask the vendor how they test for jailbreaks. Ask whether they use red teaming and what guardrails or safety classifiers sit around the model. Then try a few role-play requests yourself and see how the tool responds.