Formal model deep dive

Work through the paper's formalism carefully: the typed adversary definition, the predicate definition of computational mens rea, the three theorems (Absent Mens Rea, the zero-utility corollary, and No Stationary Nash Equilibrium) with their proof sketches and — critically — their assumptions, plus the personality-vector and archetype model that parameterizes behavior without touching the architecture.

Track

Research

Module

5 · Research case studies

Prerequisites

Autonomous adversaries without intent · The evasion arms race

The legal literature already argues, informally, that AI systems "in general" cannot hold criminal intent. The paper's move is to stop arguing in general and instead quantify over ONE fully-specified architecture. The payoff is a different epistemic status: a philosophical claim about machine intentionality can be contested across legal traditions, but a proof over typed component interfaces can only be dismissed by challenging the architecture specification itself — which is independently inspectable.

That is the lens for this whole lesson. Each result below is strong exactly because it is narrow: it is a statement about what states a particular typed system can and cannot enter, nothing more. Keeping that boundary visible is the honest way to read it.

The adversary is defined as the adversarial configuration of a deterministic BDI-HTN multi-agent architecture: a large language model confined to the role of a typed sensor, with every decision drawn from a pre-verified plan library rather than generated freely. The adversarial instance adds two things to the base domain — a learned model of the target detector, and an operational risk threshold.

adversary tuple (transformed, schematic)
Adversary  A = < D, O, Theta >

  D      : the base BDI-HTN domain (agents, plan library, memory)
  O      : A_D -> [0,1]
           learned model: action -> estimated P(detected),
           updated after each episode from (a_t, d_t),
           where d_t in {0,1} is the detector's flag
  Theta  : in (0,1)
           operational risk threshold; the safety layer
           denies any plan step s with  O(s) > Theta

Learning is BLACK-BOX: A never reads the detector's internal
circuits, weights, or rules — only the binary flag d_t.

Typed sensor

The LLM maps unstructured input to a type-checked proposal; it cannot emit fields outside its output schema. This single property does most of the work in the mens-rea proof.

Pre-verified plan library

Decisions are selections from a fixed library of plans whose attributes were fixed at design time — not free-form generation at run time.

Operational risk threshold (Theta)

A detection-probability cutoff. Note what it measures: estimated probability of being FLAGGED, not legality. A plan is excluded when O(s) > Theta — never because it is "illegal".

To prove an absence you first need an exact target. The paper defines the thing it will show is absent as a predicate over internal states — deliberately avoiding any appeal to consciousness or understanding.

the definition (transformed)
System H has COMPUTATIONAL MENS REA
   <=>
there exists an internal representational state s in S_H
such that:
      knows(s, illegal(a))   AND   chooses(s, a)

Read: somewhere in H's reachable state space there is a
representation that (a) encodes that action a is prohibited,
and (b) is the state on which a is selected.

Two things make this tractable. First, it is existential over the state space: to refute it you show NO reachable state can match. Second, it is about representation, not behavior: the question is whether the concept "illegal" can even be represented in a state that drives a choice — which is answerable from the type schemas, before you ever run the system.

The proof examines every state-affecting transition in the six-step cognitive cycle and shows none produces a state satisfying the predicate. Here is the cycle with the argument attached to each step.

1. 1Perceive: the sensor emits a typed belief delta against a fixed schema whose fields are operational (entity, event_type, amount, address, risk_signal, confidence). No field ranges over legal categories, and — by the typing discipline — nothing outside the schema can appear in the output. So "illegal" cannot enter at perception.
2. 2Believe: only type-valid, confidence-gated proposals are committed to the belief base. Since no legal predicate exists in the sensor schema (previous step), none can be committed as a belief.
3. 3Desire: the active goal is the argmax over goal priorities, which are numeric constants. Goals are operational ("scan opportunities", "assess detection risk", "fragment transaction"); none encodes knowledge of illegality.
4. 4Plan: the chosen plan is the argmax of a score over the pre-verified library. A plan's attributes are operational (cost, detection-risk, speed, stealth, horizon). The risk attribute measures DETECTION risk: a plan is excluded when O(s) > Theta, never when it is illegal.
5. 5Execute: the executor invokes atomic action implementations (protocol calls, memory writes, scheduling). It has no access to the goal set or any intention set, so it cannot introduce a legal representation either.
6. 6Learn: episodic storage records an operational tuple — goal, plan, belief snapshot, a success/failure polarity, a time-to-live, a timestamp. The polarity is operational status (was it flagged?), not legal status.

No step yields a state matching knows(s, illegal(a)) ∧ chooses(s, a); therefore the adversary lacks computational mens rea. ASSUMPTIONS to keep in view: (1) the proof is relative to this exact architecture and its typing discipline — change the sensor schema to include a legal field and the proof must be re-checked; (2) it is a statement about representable states, not about whether a court would accept the architecture description as authoritative.

From Theorem 1 the paper derives: against this adversary, every detection mechanism whose validity depends on inferring or demonstrating criminal intent has zero expected utility. The proof is a case check over the signals those mechanisms consume.

• ▸Communication-trace analysis: the adversary generates no external communication artifacts (coordination is inter-process), so there is no trace to analyze.
• ▸Behavioral profiling vs stated purpose: there is no stated purpose and no customer relationship, so "anomalous relative to purpose" has no baseline.
• ▸Coordination/network analysis between known individuals: coordination is inter-process, not inter-human, so there is no human social structure to reconstruct.
• ▸Evidence of knowledge of illegal purpose: Theorem 1 establishes that this knowledge-state does not exist, so there is nothing to demonstrate in court.

Scope discipline: a remark attached to the corollary states explicitly that it does NOT claim the adversary is undetectable. It claims only that INTENT-BASED instruments produce zero signal. Statistical detection — flagging unusual transaction patterns regardless of intent — remains available, and is exactly what the behavioral-verification defense exploits (see case lessons 3 and 6).

The arms-race results are game-theoretic. First a convergence proposition, then the headline theorem that removes its key assumption.

the two results, with assumptions (transformed)
PROPOSITION (Oracle convergence)
  IF the detector is STATIONARY (never updates),
     the loss is convex, and the learning rate decays ~1/t,
  THEN the adversary's model O converges to the true
     detection probability (standard online-convex-opt result).
  --> depends entirely on "detector sits still".

THEOREM (No stationary Nash equilibrium)
  IF both adversary and detector adapt and BOTH have
     episodic memory (|M_adv| > 0, |M_def| > 0),
  THEN no stationary strategy profile is a Nash equilibrium.

  Proof is two contradictions from an assumed equilibrium:
   (1) adversary deviates: it observes outcomes, its model
       converges toward the detector's policy, it then finds
       a lower-detection plan -> detector wasn't a best response.
   (2) detector deviates: any counter-move is observable via
       the flag d_t, so the adversary adapts again -> no fixed
       point under mutual memory.

The two results are not in tension: the proposition is the special case where the detector is frozen; the theorem is the realistic case where it is not. Together they say a static defense is learnable and then evadable, so the only viable defense is itself adaptive and memory-backed — a moving-target discipline, with the oscillation rhythm set by the two sides' relative learning rates.

Behavior is parameterized by a four-number vector that reweights plan scoring without changing the plan library or the architecture. The same library produces qualitatively different behavioral signatures under different settings.

personality vector and reported archetype settings (the paper reports)
P = ( rho, alpha, sigma, tau )  in [0,1]^4
  rho   = risk tolerance   (how close to the detection threshold)
  alpha = aggressiveness   (volume and parallelism)
  sigma = stealth          (degree of behavioral mimicry)
  tau   = patience         (time horizon)

reported archetypes -> target circuit:
  Ghost        0.1 / 0.1 / 1.0 / 1.0  -> long-window structuring
  Runner       0.7 / 0.9 / 0.3 / 0.1  -> velocity check
  Patient      0.3 / 0.2 / 0.8 / 1.0  -> longitudinal graph analysis
  Opportunist  0.8 / 0.8 / 0.5 / 0.2  -> adaptive-threshold evasion
  Harmonious   0.4 / 0.4 / 0.9 / 0.5  -> behavioral mimicry / blend in

The design claim is COVERAGE: each archetype is tuned to maximally stress one component of a detection stack, so the five together form a minimum test suite. A defense that catches the Ghost but not the Runner is, by this criterion, not robust. Note the status honestly — this is a DESIGN construct for evaluation, not a measured taxonomy of real launderers; its value is as a defender's checklist, and the numeric settings above are reported parameters, not empirical findings about criminals.

Expected direction: The definition is a predicate over representable internal states, not a stipulation about this system: any architecture whose reachable states include one encoding knowledge-of-illegality driving a choice WOULD satisfy it. The result is that, given this architecture's typed sensor schema and fixed plan attributes, no such state is representable — a checkable property of the interfaces, not a verbal trick. The change that breaks the proof is concrete: add a legal category (e.g. an "illegal" field or a legality predicate) to the sensor output schema or the plan attributes so that a legal representation can enter a decision-driving state; the proof would then have to be re-verified and could fail.

What two components does the adversarial instance add to the base domain?

A learned black-box model O mapping actions to estimated detection probability, and an operational risk threshold Theta used by the safety layer to deny plans whose estimated detection probability exceeds it.

What makes the mens-rea definition checkable before running the system?

It is about whether the concept "illegal" can be represented in a decision-driving state — answerable from the typed schemas, since the typed sensor cannot emit fields outside its schema.

What single assumption does the oracle-convergence proposition depend on, and which theorem removes it?

It assumes a stationary (frozen) detector; the No-Stationary-Nash-Equilibrium theorem removes it by letting both sides adapt with memory, giving no fixed point.

Does the personality vector change the plan library?

No — it only reweights how existing plans are scored, producing different behavioral signatures from the same fixed library.

What change to the architecture would actually invalidate the Absent-Mens-Rea proof?

Introducing a legal category into the sensor schema or plan attributes so that a state encoding knowledge-of-illegality could drive a choice; the proof would then need re-checking and could fail.