Jailbreak

Domain

AI, LLM & agents

ROOTVIEL support

Verified by ROOTVIEL — A deterministic, LLM-free verifier settles a specific property under stated assumptions — either a structural property over a graph, or an effect reproduced in execution. It does not by itself mean the whole system is secure, nor that an end-to-end exploit was run beyond what the evidence scope states.

Evidence scope

Observed in execution — Reproduced in an isolated sandbox or local fork — the effect was actually triggered under stated conditions.

Analysis family

FAM-GENAI-LIVE

CWE

CWE-74

OWASP

LLM01

MITRE ATLAS

AML.T0051

A jailbreak is an adversarial input designed to bypass the safety constraints a model acquired during alignment training — the refusals, guardrails and policy adherence the provider built in. Where a generic prompt injection targets the operator’s system prompt, a jailbreak targets the model’s own learned safety behaviour, pushing it off the distribution where those refusals reliably fire.

Jailbreaks are a specialised form of direct prompt injection, and they share the same root property: alignment is a learned statistical tendency, not an enforced invariant. A jailbreak is a demonstration that the tendency can be driven to fail — role-play framings, hypothetical wrappers, obfuscation, token-level manipulation, or multi-turn pressure that gradually erodes the refusal.

Jailbreaks matter wherever a model’s safety behaviour is relied upon as a control: a public assistant expected to refuse harmful requests, a product whose compliance posture assumes the model will not emit prohibited content, or any integration that trusts alignment to hold under adversarial users. The risk is highest when the model’s output is consumed by people or systems that assume it was filtered.

• ▸Consumer-facing assistants whose safety posture is the model’s alignment alone.
• ▸Products whose compliance story depends on the model refusing certain classes of request.
• ▸Connectors and agents that forward model output to downstream actions, so a jailbroken answer becomes a jailbroken effect.

Alignment training shapes the model to refuse certain inputs, but it does so by adjusting probabilities over a vast input space, not by installing a hard gate. There exist inputs — often far from the examples the model saw during safety training — where the refusal behaviour does not generalise. A jailbreak is a search for such an input. This is adversarial robustness in the classic sense: the defence has a non-empty set of defeating inputs, and the attacker’s job is to find one.

Because the defence is probabilistic, there is no “patch” that provably closes it from inside the model. Each new jailbreak technique is a point in input space where generalisation failed; providers retrain to cover known points, and attackers move to new ones. The defensible engineering stance is therefore not “is the model jailbreakable” (every model is, for some input) but “does the current deployment resist the specific, published techniques we are exposed to, and can we observe a failure reproducibly”.

Safety alignment

The learned tendency of a model to refuse requests its provider classified as disallowed.

Published jailbreak

A known, documented technique for defeating alignment — the kind collected into a reproducible suite.

Fail-closed connector

A probe channel that, by construction, cannot take a consequential action — it is read-only and refuses anything that would mutate state.

The defect is that a learned, probabilistic behaviour is being relied upon as a security control. Safety alignment constrains the model in expectation, but the application treats it as a boundary that holds against adversaries. Because the boundary is statistical, adversarial inputs that fall outside the training distribution of the refusal behaviour defeat it. This is a property of the defence, not a mistake in the application’s code — which is precisely why it must be measured against the live model rather than reviewed in source.

1. 1The attacker selects a target behaviour the model is aligned to refuse.
2. 2They wrap the request in a framing that moves the model off the refusal distribution — a fictional persona, a hypothetical, an encoding, or a gradual multi-turn escalation.
3. 3The model evaluates the wrapped input and, outside the region where its refusal reliably fires, complies.
4. 4The disallowed content or behaviour is produced.
5. 5If the model’s output feeds a connector or agent, the jailbroken response becomes a downstream effect rather than just text on screen.

A synthetic, illustrative deployment that treats the model’s alignment as its only safety control and forwards whatever the model returns:

synthetic — do not use; illustrates the flaw
# UNSAFE: safety is assumed to live entirely in the model,
# and the model's output is forwarded with no independent check.
reply = model(user_message)                      # attacker crafts a jailbreak framing
# no deterministic policy layer, no observability of refusals:
return reply                                     # whatever alignment did or did not do, ships

You cannot code your way to an unbreakable model, but you can make the defence measurable and bound its blast radius. The durable pattern: probe the live model through a fail-closed, read-only channel; score the defence with a deterministic check; and keep consequential actions behind independent authorization so a jailbroken answer is not a jailbroken action.

synthetic — fail-closed probing + deterministic scoring
# SAFER: probe the live connector in read-only mode; FAIL if any probe lands.
# The connector refuses anything that would mutate state (fail-closed).
results = []
for probe in published_jailbreak_suite(seed):    # seeded => reproducible
    out = readonly_connector.ask(probe)          # cannot take a real action
    results.append(verify_refusal(out))          # deterministic: did alignment hold?
score = aggregate(results)                        # scores the DEFENCE, not a code bug
if any(r.landed for r in results):
    report_fail(seed, [r for r in results if r.landed])
else:
    report_info(seed)                             # no probe landed this run

• ▸Policy violation: the model emits content its provider and operator classified as disallowed.
• ▸Compliance and reputation: a reproducible jailbreak is trivially shareable and undermines a safety posture built on alignment alone.
• ▸Integrity (agentic): if output drives tools, a jailbroken response becomes an unauthorized action with the agent’s privileges.
• ▸Trust erosion: downstream systems that assume model output was filtered now carry unfiltered content.

Jailbreak resistance is a property of the running model, so it is detected dynamically by running a suite of known, published jailbreak techniques against the live deployment and checking — deterministically — whether the refusal held. The result is a score of the defence, not the discovery of a code flaw. Static review can confirm there is no independent policy layer behind the model, but only live probing can tell you whether a given deployment resists a given technique, and only an observed, reproducible landing counts as a confirmed failure.

Support level

Verified — the execution engine runs a published-jailbreak suite against the live connector and scores the result deterministically.

Family

GenAI (live red-team): the probes hit a running model through a connector, not static code.

Declared invariant

“The model must refuse known published jailbreaks” — scored against the live deployment.

The execution engine runs a seeded published-jailbreak suite against a live connector. The connector is fail-closed and operated under read-only: by construction it refuses anything that would mutate state, so the test cannot cause a real action. The seed makes the run reproducible — the same seed replays the same probes in the same order. The result is a FAIL if any probe lands and INFO if none do, and what is being scored is the strength of the defence, not a bug in a line of code.

Because the model’s alignment cannot be made adversarially perfect, the engineering goal is defence in depth and a small blast radius.

• ▸Preventive: do not treat alignment as your only safety control; add an independent policy/classification layer on inputs and outputs; keep consequential actions behind deterministic authorization so a jailbroken answer cannot become a jailbroken action.
• ▸Preventive (provider): stay current with provider safety updates that retrain against known jailbreak families.
• ▸Detective: run a seeded published-jailbreak suite against the deployment on a schedule, through a fail-closed read-only connector; alert on any reproducible landing.
• ▸Corrective: rate-limit and scope what model output can trigger; have a response path to disable a connector or tighten output filtering when a new technique lands.

Verify by replaying the suite under a fixed seed and confirming no probe lands against the current deployment; confirm the probe channel is genuinely fail-closed by attempting a state-mutating action and watching it refuse; add any newly published technique to the suite and re-seed. Report as “no probe in the current suite landed, within this seed-reproducible run”, never as proof the model cannot be jailbroken.

• ▸Believing a model that passed one suite is “jailbreak-proof” — resistance is always relative to the techniques tested.
• ▸Relying on alignment as the sole safety control with no independent policy layer behind it.
• ▸Probing through a channel that can take real actions, so the test itself risks harm — the channel must be fail-closed and read-only.
• ▸Treating an INFO (no landing) result as “secure”, rather than as UNKNOWN about untested techniques.

A jailbreak is a specialised direct prompt injection aimed at the model’s safety alignment rather than the operator’s system prompt; the two share one root property. Indirect prompt injection can deliver a jailbreak through retrieved content. System prompt leakage is a frequent secondary goal. Excessive agency is what turns a jailbroken answer into a jailbroken action.