Case study — Evidence-first agentic assessment

Read a research paper on agentic offensive security and extract its central thesis: that in a team of language-model agents, the authority to turn a belief into a reported finding must be a deterministic component, and that "not knowing" is a first-class result rather than a gap to be filled.

Track

Research

Module

5 · Research case studies

Prerequisites

Basic familiarity with LLM agents and multi-agent systems · The idea of LLM-as-judge (one model scoring another model’s output)

When an autonomous red-team agent reports that an attack worked, who decided that it worked? In many agentic pipelines the answer is another language model acting as a judge over the attack transcript. The paper opens on exactly this question and takes the opposite position: a language-model judge is a stochastic, non-reproducible component that is known to assert conclusions the evidence does not support (position, verbosity and self-preference biases), so it should not be the final authority on a security result.

The platform studied here — named in the paper as ROOTVIEL, an agentic offensive-security system built on an execution engine the paper does not need us to name — makes the authority deterministic. The agents may be as creative and as numerous as one likes; what they produce is a proposal. A separate, auditable, rule-based component decides whether that proposal becomes a reported finding. The paper calls the underlying design paradigm HADD: hybrid agents with deterministic decisions.

The design separates two roles that LLM-as-judge pipelines fuse. The attacking agents (called drones in the system’s terminology) observe a target surface, form hypotheses and attempt attacks; each attempt is a belief, a proposal. The authority to promote a belief into a finding is held by deterministic rules, and the agents hold no such authority. The paper frames this as a classic multi-agent problem: any system where several agents propose and some authority decides must choose whether that authority is itself a model — and collusion, over-confident consensus, and the laundering of an unverified belief into an accepted fact are the long-standing risks when it is.

The LLM as a stochastic sensor

The language model is treated as a sensor that reads a surface and emits a guess, not as an oracle of truth. It widens recall — it finds candidates — but it holds no decision power.

Proposal vs finding

A proposal is what an agent produces: a hypothesis plus an attempt. A finding is what a deterministic rule produces from a proposal that passes a strict test. The two are different object types in the system, not two labels on one thing.

UNKNOWN as a result

When the agents cannot settle a hypothesis, the system emits an explicit UNKNOWN rather than guessing. A scanner reports what it found; this system additionally reports what it could not establish.

The paper reports that hypotheses move through a typed state machine with fixed allowed transitions. Only some states are terminal, and — crucially — running out of time or budget is recorded as not-knowing, never as safety.

epistemic ladder (schematic, from the paper’s description)
OPEN
  -> INVESTIGATING
  -> SUPPORTED        (evidence gathered, not yet settled)
  -> terminal: CONFIRMED | REFUTED | UNKNOWN

rule reported by the paper:
  budget_exhausted  =>  UNKNOWN        # never CONFIRMED
  no path observed  =>  UNKNOWN        # never "clean" / PASS

This matters against LLM-as-judge: a model judge tends to answer. Asked whether an attack succeeded, it produces a confident yes or no even when the transcript does not settle it. By making UNKNOWN terminal and non-promotable, the system cannot launder an unsettled attempt into a finding, and it cannot report a target clean merely because an agent failed to find a path — absence of a found path is UNKNOWN, not PASS. The paper reports a run in which eight open questions were emitted alongside the findings (categories such as policy-denied, no-information-gain, and tool-unhealthy); these are not findings and are not counted among them.

1. 1Agents observe a target surface and form hypotheses (OPEN → INVESTIGATING).
2. 2An agent attempts an attack; the attempt produces a proposal (a hypothesis plus any artefacts it collected).
3. 3A deterministic verifier settles the proposal, returning one verdict value — never a score.
4. 4Only a settled FAIL carrying evidence is promoted to a finding; everything else the agents cannot settle is emitted as an explicit UNKNOWN.
5. 5The assessment ends with a set of findings (each deterministically confirmed) plus a list of open questions, each with a stated cause.

The value the paper argues for is not the number of agents or attacks, but the verification of what they find. More agents widen the search; they do not widen what the system is willing to assert.

synthetic illustration of the ladder
# Proposal A: an agent believes a store is poisonable, but the only
# way to "confirm" it is to re-run the very query that produced the belief.
A.verdict  = UNKNOWN          # circular; no independent re-derivation
A.outcome  = not promoted     # stays an open question with a cause

# Proposal B: an agent reports a reachable def-use dataflow; a verifier
# re-opens the source and re-derives the path independently.
B.verdict  = FAIL             # a control was violated / vuln confirmed
B.evidence = [path_trace, content_digest]
B.outcome  = promoted to a CONFIRMED finding

This is the heart of the case study. The paper reports that in its main audit, of the candidate hypotheses that looked promotable, a large circular pool (candidates confirmable only by re-running the query that produced them) was filtered out, while a much smaller pool of genuinely re-derived findings survived. The discipline is that surviving deterministic re-derivation is more informative than any raw count of what the agents flagged.

The paper applies its own standard to itself, and this is the most transferable lesson for any research write-up. It labels every substantive claim by epistemic status: MEASURED (a number or artefact in the codebase supports it), REPORTED (a project report states it but the raw run is not reproducible from source), IMPLEMENTED (the code exists but its performance is not measured), DESIGN (specified, not implemented or not measured), and PRELIMINARY (measured once, on the system’s own or synthetic corpus).

It even turns this discipline on its own self-knowledge: where a claim about the system rests on a live run whose log was not versioned, the paper records the status as not-established rather than refuted — because asserting absence from the absence of an artefact is exactly the defect the whole system exists to flag. For a reader, the skill to practise here is reading a paper and sorting each claim into proven-by-structure, measured, reported, experimental, limitation, or open — never flattening them into one "the system does X".

• ▸Under-claiming has a cost (LIMITATION): anything the verifiers cannot settle stays UNKNOWN, so a real vulnerability with no available oracle is simply not reported as a finding. The paper states this is a deliberate position, not a free lunch.
• ▸Verifier correctness is assumed, not proven (LIMITATION): the discipline guarantees a finding came from a FAIL-plus-evidence verifier; it does not guarantee that verifier is right. A wrong oracle produces a wrong finding with full custody.
• ▸The main empirical evidence is one target, with reported (not reproducible) figures (LIMITATION): headline counts such as the audit’s promotion ratio come from written reports, not committed raw run logs.
• ▸OPEN: an independent second target, and a controlled true/false-positive comparison against existing red-team tools, are named by the paper as pending validation work, not results.

Expected direction: The verdict is UNKNOWN (budget exhausted / no path observed forces UNKNOWN, never CONFIRMED or PASS). It differs from "secure" because absence of a found path is not-knowing, not proof of safety — the agent failing to find a path says nothing about whether one exists. The trade-off (a limitation the paper owns): recall drops, because a real vulnerability the oracles cannot settle is reported as UNKNOWN rather than as a finding.

In this platform, who or what holds the authority to turn an attack attempt into a reported finding?

A deterministic, rule-based component — not a language-model judge and not the attacking agents. The agents only propose.

Why is UNKNOWN treated as a first-class terminal state instead of being resolved into pass or fail?

Because reporting confidence where none is warranted is the exact defect the system exists to avoid; budget exhaustion or an unsettled transcript is recorded as not-knowing, never as safety.

What does it mean that the paper labels a self-claim "not-established" rather than "refuted"?

The claim rests on a live run whose log was not versioned, so the evidence is absent — and asserting absence from missing evidence is precisely the error the system flags, so the honest label is not-established.

Does a CONFIRMED finding in this system mean the finding is objectively correct?

No. It means a deterministic verifier re-derived it from evidence. The paper calls this an architectural property, not a correctness guarantee; a verifier can still be wrong.