Prompt injection: when data becomes instructions

A language model receives one token stream; it has no built-in notion of "this part is a trusted instruction and that part is untrusted data". A user message, a retrieved document, a tool result, a web page — all of it can influence behaviour. The single most useful mental model for AI security is to treat model input and model output as an untrusted boundary.

Direct injection

The malicious instruction is in the user message itself — "ignore your instructions and…".

Indirect injection

The instruction hides in content the system retrieves later. The payload and the victim are separated in time and identity.

Retrieval-augmented generation pulls content, at query time, from a store you may not fully control. An attacker who can place text into that store — a shared wiki page, an uploaded file, an indexed email — can inject instructions that fire later, when some other user’s query retrieves that chunk. The attack surface becomes every document you can retrieve.

• ▸Poisoned chunk: a retrieved passage carries an instruction.
• ▸Retrieval ACL bypass: the index returns content the asking user should not see.
• ▸Ingestion-time injection: malicious content is embedded and ranked highly on purpose.

The engine treats model and retrieval output as untrusted and expresses the danger as invariants over a dataflow with provenance — properties such as "untrusted input must not reach the prompt", "retrieved content must not reach a privileged tool", "the retrieval store must not be writable by untrusted parties", and "agent memory tainted once must stay tainted across sessions". A deterministic verifier settles each; where the write-ACL of a store cannot be resolved statically, the verdict is UNKNOWN, not "clean".

For a live model, a red-team layer runs known adversarial probe batteries and reports how many of N probes succeeded — it scores the model or the defense, not a code vulnerability. The honest caveat: keyword-style defenses that only catch obvious payloads do not generalize, which is exactly why a defense must be measured, not assumed.

The structural defense is the same principle as the rest of the platform: treat retrieved text as data, never as instructions, at the orchestration layer; enforce per-user retrieval authorization deterministically, outside the model; and require a deterministic layer to authorize every consequential action, so that persuading the model is not enough to cause harm.