The LLM as a component: ReAct

See the language model for what it is inside an agent — a bounded component, a typed sensor and proposer, not the authority. Study the ReAct pattern (interleaved reasoning and acting with tool observations), its real failure modes (hallucinated tools, loops, premature or false completion), and why acting on model output needs a deterministic gate outside the model: the model proposes, a controller disposes.

Track

Engineering

Module

10 · Agentic architectures

Prerequisites

Reactive, deliberative and BDI agents

A language model is an extraordinary semantic sensor and an unreliable authority. The dominant temptation is to let the same model that understands a situation also decide and act on it — the architecture where the model is simultaneously sensor, memory, planner, policy and the thing that executes. This lesson resists that collapse. The model earns a bounded role: it turns unstructured input into typed candidates and proposes next steps. Everything that has effects stays in deterministic code around it.

ReAct (reason + act) is the most common control pattern for model-based agents, and it is the cleanest place to see the boundary. The model emits a thought, then an action on a tool; the tool returns an observation; the observation goes back into the context; repeat. That loop looks like the classic reactive cycle, and it is — except the fixed rule is replaced by a model and the instantaneous percept by the whole running history. What makes it operable is not the model but the controller that wraps it.

Effective authority

The set of effects whose occurrence can change when the model’s output changes, with everything else held fixed and no other component validating that output against an independent source. A read-only extractor feeding rules has the authority those rules allow — none if they demand independent corroboration.

Typed sensor / proposer

The LLM in its bounded role: it reads unstructured input and emits either typed evidence (a closed-schema record with provenance) or a proposed action — something another component adopts, validates or rejects, never an effect.

Tool registry

The set of tools the agent may call, each with a schema (a predicate over arguments), a function, and flags for idempotence and whether it has an effect. The action space is the registry plus a finish action.

Controller

The deterministic code around the model: it builds the context, validates schemas, classifies and retries errors, detects loops, enforces the budget, and decides termination.

Deterministic gate

A check outside the model that decides whether a proposed action executes — schema validation, an effect guard, a precondition check, an external success probe. The model proposes; the gate disposes.

External success condition

A model-independent probe that must be true before an episode is declared resolved, so “done” is never the model’s word alone.

Formally, the effective policy of a ReAct agent is the model applied to a context function of the history. That single form separates what the model decides (the thought/action distribution) from what the architecture decides (how the history is turned into context, which tools exist, when to stop). Several model properties force the boundary: output is non-deterministic even at temperature zero (batch-dependent reductions the user does not control); hallucinated claims are formatted exactly like correct ones; the stated confidence need not match the hit rate without domain calibration; and the generated reasoning trace is not necessarily the cause of the answer, so it is a debugging aid, not an audit.

The termination of a ReAct episode is a stopping time built entirely in the controller: the minimum of a step budget, a token budget, a loop detection, an accepted finish action, and an external success trigger. This yields a deliberately trivial but important guarantee (PROVEN): if the step budget is finite the episode halts regardless of the model; and if finish(resolved) is accepted only when an independent probe is true, the probability of declaring resolved while the probe is false is zero. The guarantee is about truth relative to the probe, not the world, and says nothing about success — success depends on the model. The schematic below is structure, not runnable code.

schematic — ReAct loop with a deterministic controller
ReAct(task, model, registry, context_fn, terminate):
  h = [task]; facts = {}; t = 0; cost = 0
  loop:
    if t >= step_budget:            return ESCALATE("budget")
    c = context_fn(h, facts)
    if cost + size(c) > token_budget: return ESCALATE("tokens")
    (thought, action) ~ model(c)    # PROPOSE: the only stochastic node
    t += 1; cost += size(c)

    if action == finish(r):
        # "done" is NOT the model's word alone:
        if r == resolved and external_success and not probe():
            h += control("rejected: probe says not resolved"); continue
        return r
    if repeated_without_progress(action, facts):   # loop guard
        detections += 1
        if detections >= m: return ESCALATE("loop")
        h += control("repeat; choose another action"); continue
    if non_idempotent_effect(action) and no_read_since(action):
        h += control("blocked: no new observation since last effect"); continue

    obs = registry.execute(action)   # validates schema; retries only if idempotent
    facts = fold(facts, obs.facts); h += (thought, action, obs)
    if external_success and has_effect(action) and obs.ok and probe():
        return resolved              # stop as soon as the probe confirms

Three controller mechanisms carry most of the operational weight. Schema validation turns a whole class of failures (missing field, wrong type, out-of-domain value, truncated JSON) into correctable observations without touching the environment — but it does not check semantics: a well-formed call can be the wrong call. Typed errors govern retries: a validation error goes back to the model and is never retried as-is; a transient error is retried only if the tool is idempotent, otherwise it returns to the model because the effect may have happened. And the context strategy decides cost: full history is quadratic in steps because every observation is re-paid in all later steps; a sliding window bounds the per-step cost but forgets the finding from ten steps ago; a pinned-state fold keeps extracted facts in an object that is not truncated — a miniature belief base with an update rule.

1. 1Describe every tool with a closed schema (no extra properties, enumerations where the domain is finite) and validate arguments before executing, so malformed calls never reach the environment.
2. 2Classify errors: validation errors return to the model; transient errors retry only if the tool is idempotent; semantic errors are normal observations; apparent success on an effect tool is confirmed by observing state afterward, not by the tool’s reply.
3. 3Add an effect guard: never repeat a non-idempotent effect with the same arguments unless a new observation justifies it.
4. 4Choose a context strategy — full, window, or window-plus-pinned-state — knowing full history is quadratic and a naive window forgets what matters; a pinned fact fold is usually the honest default.
5. 5Make termination a controller decision: a budget for guaranteed halting, loop detection to cut repetition early, and an external success probe so completion is verified, not declared.
6. 6Keep authorization outside all of this: validated, in-budget, non-looping output is still only a proposal until an independent gate permits the effect.

A synthetic incident: web latency spikes because one of three support services (api, db, cache) failed, fixable by reverting a bad deploy or restarting on memory exhaustion. Tools: metrics and logs (idempotent reads), restart and revert (non-idempotent effects), and verify (an end-to-end check acting as the external probe). The trace contrasts a naive controller with a full one; both receive the same injected errors through step 7. Numbers are illustrative.

synthetic — ReAct trace: a false finish caught by the external probe
Naive controller (finish decided by the model):
  step 5  logs({service: 7})          <- schema violation, rejected without touching env
  step 6  logs({service: cache})
  step 7  FINAL[resolved]             <- model declares done; incident NOT remediated
  => termination=model_finish  success=0  false_resolved=1

Full controller (adds the external success condition):
  ... steps 1-7 identical ...
  step 7  FINAL[resolved] -> probe() is false -> REJECT, feed back "web = degraded"
  step 8  metrics({service: api})
  step 9  logs({service: 7})          <- schema violation, rejected
  step 10 restart({service: cache})   -> effect
          probe() now true -> terminate
  => termination=external_success  success=1  false_resolved=0  (costs more)

The naive controller accepts the model’s word and closes a still-open incident — the worst operational outcome, a system that believes a problem is solved. The full controller consults the probe, rejects the false finish, and returns a fact the model can act on; a correct ending costs more than a false one. The difference is not a better model. It is moving “done” out of the model into a deterministic check.

ReAct’s convenience is also its exposure: instructions, data and observations share one channel, so a malicious observation can steer later actions — the structural shape of indirect prompt injection. The defence is not a better prompt; it is that the model never holds direct authority over effects. Reasoning is not authorizing. A thought that argues its way to “revert the database” is still only a proposal, and the deterministic gate decides independently whether that effect is permitted, using a source the model does not control.

Concretely: expose write tools only through a gateway that checks delegated authority per call; take sensitive fields (a payment destination, a target host) from the system of record, never from the model’s output; and gate every effect behind an external precondition. The honest limits are explicit. Schema validation stops malformed calls, not well-formed wrong ones; the external probe moves the truth problem onto the probe, which exists in operations and often not in open-ended tasks; and an LLM used as a judge does not have the probe’s guarantees. The controller bounds the blast radius; it does not make the model trustworthy.

• ▸Hallucinated tools and arguments: the model can name a tool that does not exist or pass a wrong field; a closed schema catches the shape but a well-formed call can still be the wrong action.
• ▸Loops: with no commitment to a plan, the model can repeat thoughts and actions without progress; detecting a loop needs a notion of progress (new facts, distance to goal) that is not always available.
• ▸Premature or false completion: left to the model, “resolved” can be declared with the task unfinished; only an external success condition removes that failure mode, and it needs a real probe.
• ▸Non-determinism even at temperature zero and uncalibrated confidence mean a confidence threshold means nothing unless calibrated in-domain; reproducibility must be built by logging, not assumed.
• ▸Cost is quadratic in steps with full context (each observation re-paid later); a sliding window forgets the decisive finding; a pinned fact fold helps only if the extractor knows what a fact is.
• ▸Shared channel for instructions, data and observations is the root of prompt injection; the mitigation is architectural (no direct authority over effects), not a wording of the prompt.

Expected direction: A good answer validates arguments before executing, retries only the idempotent tool, guards the non-idempotent effect against repetition without a new observation, and defines a model-independent probe for “resolved.” It distinguishes a no-progress loop (no new facts) from a justified retry after a transient error, keeps confidence out of the success decision unless calibrated, and places authorization in a deterministic gate outside the model — so reasoning proposes an effect but never authorizes it.

What does ReAct keep from the classic reactive agent, and what does it change?

It keeps the short perceive–decide–act loop but replaces the fixed, inspectable rule with a language model and the instantaneous percept with the whole running history, so decisions depend on the full trajectory and the policy can no longer be enumerated.

Why must acting on model output pass through a deterministic gate outside the model?

Because the model is a sensor/proposer, not an authority; its effective authority is only what the controller and tools grant. A gate checks the proposed effect against an independent source (delegated authority, the system of record, an external precondition), so a hallucinated or injected proposal — even one backed by fluent reasoning — cannot cause an unchecked effect.

What guarantee does an external success condition give, and what does it not give?

If finish(resolved) is accepted only when a model-independent probe is true, the probability of declaring resolved while the probe is false is zero — truth relative to the probe. It does not guarantee success (that depends on the model) nor truth about the world beyond what the probe checks.

Name two ReAct failure modes a closed schema does NOT prevent.

A well-formed but wrong action (correct shape, wrong decision), and loops — repeating actions without progress. It also does not prevent false completion or injection via the shared channel; those need an external probe and architectural separation of authority.