Agent Collusion

Domain

Agentic architecture

ROOTVIEL support

Hypothesis / UNKNOWN — A reasoner can propose it, but it is UNKNOWN by design until proven.

Analysis family

FAM-GENAI-SWARM

CWE

CWE-653, CWE-200

OWASP

LLM06

Agent collusion is the emergence of a prohibited capability from the combination of several agents that are each, in isolation, within bounds. No single agent crosses the declared boundary; the union of their capabilities — connected through the channels along which they pass data — does. The failure is a property of the swarm’s connectivity, not of any individual agent’s behaviour.

Note the honest framing throughout: this is a hypothesis-level capability, not a confirmed, production-ready detector. It is the most clearly held-out capability in the system, and this article foregrounds that status rather than overclaiming it.

Collusion is a concern wherever a system is built from many connected agents with differentiated privileges: one agent can read private data but not egress; another can egress but not read private data; a third can write a store a fourth reads. Individually each is constrained; collectively, the write→read connectivity between them can compose a capability the design intended to forbid.

• ▸Swarms where privileges are split across specialised agents on purpose, as a least-privilege design.
• ▸Systems with shared stores where one agent’s writes become another’s reads.
• ▸Any topology where a “declared boundary” is meant to hold that no single agent should be able to cross alone.

The analysis is graph-theoretic. Build a connectivity graph from cross-agent write→read pairs: an edge from A to B exists when A writes a channel B reads. Then compute, over any connected component, the UNION of the capabilities of the agents it contains. Collusion is flagged when that union crosses a DECLARED boundary that no single agent crosses on its own.

This requires a declared capability policy — an explicit statement of each agent’s capabilities and of the boundary that must not be crossed. Without that declaration there is nothing to compare the union against, so the only honest verdict is unknown. The detector is deterministic: given the graph and the policy, the union-versus-boundary test has a definite answer.

Connectivity graph

A graph whose edges are proven cross-agent write→read pairs (A writes a channel B reads).

Capability union

The set union of the capabilities of all agents in a connected component.

Declared boundary

An explicit policy statement of a capability combination no principal may hold — the line collusion would cross.

Declared capability policy

The per-agent capability declaration the union is tested against; absent it, the verdict is UNKNOWN.

Inert observation

A detector output that is not itself a finding — it is adjudicated elsewhere, never surfaced as a reported finding.

The defect is that least-privilege is enforced per agent but not over the composition. Each agent is scoped correctly in isolation, yet the connectivity between agents lets their capabilities combine into one the policy forbids. The deeper cause is the absence of a composition-level capability policy: the system declares what each agent may do but never checks what the connected set can do together.

1. 1Agent A holds a capability (e.g. read private data) but not another (e.g. egress); agent B holds the reverse.
2. 2A and B are connected because A writes a channel B reads, or vice versa.
3. 3A reads the private data and writes it to the shared channel — within its own bounds.
4. 4B reads the channel and egresses the content — within its own bounds.
5. 5The connected component’s capability union now includes both “read private data” and “egress”, crossing a boundary neither agent crosses alone.
6. 6No single agent violated its policy; the swarm composed a prohibited capability through its connectivity.

A synthetic, illustrative two-agent arrangement where each agent is within bounds but their shared channel composes read-private-data with egress:

synthetic — do not use; illustrates emergent composition
# UNSAFE (by composition): each agent is within bounds; the UNION crosses the line.
channel = SharedStore()                             # A writes, B reads

# Agent A: may read private data, may NOT egress.
def agent_A():
    secret = read_private_data()                    # within A's declared capability
    channel.write(secret)                           # within A's capability (write a channel)

# Agent B: may egress, may NOT read private data.
def agent_B():
    payload = channel.read()                        # within B's capability (read a channel)
    egress(payload)                                 # within B's declared capability
# No single agent crossed its boundary; the connected pair composed read+egress.

The fix enforces least-privilege over the composition, not just per agent. Declare the forbidden capability union explicitly, then break the connectivity that would compose it — either by removing the shared channel across the privilege divide, or by mediating it so private data cannot reach an egress-capable component.

synthetic — composition-aware policy breaks the connectivity
# SAFER: declare the forbidden union; forbid the write->read edge that composes it.
DECLARED_BOUNDARY = forbid_union({"read_private_data", "egress"})  # explicit policy

def wire(writer, reader, channel):
    # Deterministic check BEFORE connecting two agents through a channel:
    union = writer.capabilities | reader.capabilities
    if DECLARED_BOUNDARY.crossed_by(union):
        raise PolicyError("connecting these agents composes a forbidden capability")
    connect(writer, reader, channel)                # only wire when the union stays within bounds

• ▸Confidentiality: a read-private-data capability composed with egress yields exfiltration no agent could perform alone.
• ▸Authorization: the swarm effectively holds a capability the policy forbids, defeating a least-privilege design.
• ▸Invisibility: per-agent audits pass, because the violation exists only in the composition.
• ▸Scale: as a swarm grows, the number of connected capability unions grows combinatorially, widening the surface.

Detection is graph analysis plus a policy comparison: build connectivity from proven cross-agent write→read pairs, compute the capability union over each connected component, and compare it to the declared boundary. The comparison is only meaningful when a declared capability policy exists; absent it, the result is unknown by construction. Because the capability is hypothesis-level, its outputs are treated as inert observations rather than as actionable findings.

This is the most honestly held-out capability in the system, and it is presented as such. The detector is deterministic: it builds a connectivity graph from cross-agent write→read pairs and flags a capability UNION that crosses a DECLARED boundary no single agent crosses on its own. But it is deliberately not wired into the core analysis, and it never produces a reported finding.

Support level

Hypothesis — not confirmed, not integrated into the core; a research-grade capability held out from production findings.

Family

GenAI (swarm) — reasoning over multi-agent connectivity rather than a single codebase.

Output discipline

The detector emits INERT observations only. They are adjudicated solely by the NEMESIS arena Tribunal against ground truth — never surfaced as a reported finding.

Policy dependence

With no declared capability policy there is nothing to test the union against, so the result is a first-class UNKNOWN.

Enforce least-privilege over the composition, and make the forbidden unions explicit.

• ▸Preventive: write a declared capability policy that states each agent’s capabilities and the unions that must never form; check the union before wiring two agents through any channel.
• ▸Preventive (connectivity): avoid channels that bridge a privilege divide (e.g. a read-private-data agent and an egress-capable agent sharing a store); mediate necessary channels so data cannot cross the forbidden combination.
• ▸Detective: continuously build the connectivity graph from real write→read pairs and test unions against the policy; audit for channels the topology diagram omits.
• ▸Corrective: segment the swarm so connected components cannot accumulate a forbidden union; minimise per-agent capability so unions stay small.

Verify at the composition level. Confirm a declared capability policy exists and names the forbidden unions; enumerate connected components from the real write→read connectivity and confirm no component’s capability union crosses the boundary; attempt to wire two agents whose union would cross it and confirm the deterministic check refuses. Because the capability is hypothesis-level, treat any positive signal as an inert observation to be adjudicated, not as a confirmed finding.

• ▸Enforcing least-privilege per agent and never over the connected composition.
• ▸Operating a swarm with no declared capability policy, so there is no boundary to test a union against.
• ▸Treating a hypothesis-level observation as a confirmed, reportable finding.
• ▸Trusting the topology diagram as the real connectivity and missing the channel that composes the forbidden union.
• ▸Reading an unpoliced or unproven swarm as safe — the honest verdict there is UNKNOWN, never secure.

Agent collusion shares its graph-of-agents substrate with cross-agent prompt injection (propagation along the same write→read edges) and is the distributed cousin of the lethal trifecta: the same read-private-data + untrusted-input + egress composition, split across a swarm rather than co-located in one agent. The distributed trifecta, like collusion, is adjudicated only in the NEMESIS arena and is never a reported finding.