ZK in agents: the evaluation, measured in detail

Read the case study’s experiments the way an engineer evaluating a prototype should: understand the measurement setup and its load conditions, walk each measured question in detail with figures attributed to the paper, keep the proven-vs-tested-vs-measured labels crisp, and take seriously the ablations, confounds and failure analysis the paper itself foregrounds.

Track

Research

Module

5 · Research case studies

Prerequisites

ZK in agents: architecture & algorithms, step by step · ZK in agents: results, limits and open problems

Before any number, fix the vocabulary. The paper gives mechanisms and TESTS rather than formal proofs, so nothing here is “proven” in the theorem sense. Correctness properties are TESTED (a suite constructs a violating input and asserts rejection). Everything with a unit attached — bytes, milliseconds, gas, messages per second — is MEASURED on the prototype under stated conditions. Some measured results are of a FRAGMENT of the check, and some are SYNTHETIC (deterministic, run without a live multi-organization system).

Tested (correctness)

~46 tests across seven suites, including dedicated invariant tests that build a widening hop, a tampered signature, an expired credential and a substituted credential and assert rejection. Evidence bounded by the constructed cases — not a proof.

Measured (performance)

Sizes, generation/verification times, gas, throughput. All on one host, under load. Reported with p50 and p95 for the lanes with enough samples.

Measured-but-fragment / synthetic

The hash-based and folding adapters are measured on a FRAGMENT (attenuation + linkage, no signatures). Messaging and transparency figures are synthetic: they characterize cryptographic and structural cost, not the behaviour of a deployed population.

The paper reports all figures from a single desktop-class host (a modern many-core consumer CPU, on the order of two dozen cores and ~125 GiB RAM, Linux), with a standard zero-knowledge toolchain (a circuit compiler, a proving library, plus two native prover binaries in another systems language) and powers-of-tau parameters at modest sizes. The methodological choice that colours everything: the host was deliberately NOT idle.

• ▸The host permanently ran other container stacks (an event broker, a document store, a vector store, a log aggregator, a search backend) — treated as the normal condition, not an accident of scheduling.
• ▸The paper reports a load-sensitivity factor of about 2.1 (a single verification moving from roughly 0.58 ms to 1.23 ms as all cores saturate), and reports the load average alongside each result so a reader can situate it.
• ▸Consequence: every figure is best-effort under load, not best-case in isolation, and should be read as pessimistic by roughly a factor of two. A reader on a quiesced machine should expect better.
• ▸Sample sizes differ by driver and are tabulated rather than aggregated — e.g. hundreds of samples for the transparent baseline, tens for the pairing-based lanes and the delegation circuit, a couple of thousand for scope levels, and on the order of 10^3–10^5 for the reception and non-repudiation paths. Two native-binary adapters report a single value rather than percentiles.

How do proof size and timing scale with chain depth? The paper reports a sharp split between lanes.

• ▸Transparent baseline: proof size grows with depth and discloses the witness in full — the paper reports hundreds of bytes at shallow depth rising to tens of kilobytes by depth ~32 (e.g. ~0.5 kB at depth 1 up to ~18.6 kB at depth 32). This is the zero-cryptography reference point, not a competitor.
• ▸Pairing-based lanes (groth16, plonk): constant proof size regardless of depth — the paper reports roughly 722 B and ~2100 B respectively — with verification roughly flat at single-digit milliseconds.
• ▸Generation is where the pairing-based lanes separate: one stays in the tens of milliseconds as depth grows while the other rises into the hundreds-to-thousands of milliseconds, which the paper attributes to the price of a universal ceremony rather than to depth per se.
• ▸Folding adapter: constant proof size (the paper reports ~11,848 B) whether 4, 8, 16 or 32 hops, with verification of tens of milliseconds — but it proves the FRAGMENT (attenuation + linkage), not the full check.

Three switchable scope levels were measured per hop at depth 8 over ~2000 samples: a flat action set, prefix/wildcard patterns, and numeric constraints. The headline is counter-intuitive and is one of the paper’s named lessons.

• ▸Flat action set: the cheapest — the paper reports on the order of 0.08 µs per check.
• ▸Numeric constraints: the paper reports on the order of 0.29 µs — more expensive than flat but modest.
• ▸Prefix/wildcard patterns: the MOST expensive — the paper reports on the order of 4.9 µs, i.e. roughly 59× the flat level and ~17× the numeric level above it.

The anomaly: cost is NOT monotone in expressiveness. By a count of distinct expressions the workload admits, the pattern level is the LEAST expressive of the three yet the most expensive, because cost follows the DECISION PROCEDURE (pattern subsumption requires comparing structures pairwise; numeric constraints reduce to interval containment, which is arithmetic), not the intuitive expressiveness ordering. The paper reports it found this only because measurement contradicted an assumption nobody had stated.

Q3 (non-repudiation and batching): per-message sender signing is reported constant at ~0.039 ms across every batching window from 50 ms to 5 s. Enlarging the window amortizes the operator signature and grows the inclusion proof logarithmically, while anchoring latency grows linearly toward current validity. So the real trade-off is not cost-against-cost but amortized cost against time-to-finality — the axis a deployment actually chooses on.

Q4 (equivocation): an operator presenting divergent views is detected whenever at least two witnesses sit on different branches. Detection latency equals the gossip interval and is independent of registry size; registry cost scales with the witness population instead — the paper reports roughly 22.5 kB of roots for four witnesses and ~60.9 kB for sixteen. A finer gossip interval buys lower detection latency, not lower bytes. Both are SYNTHETIC measurements of structural cost.

Measured on two adapters, this asks whether to prove a chain as one monolithic statement or as composable per-segment proofs. The answer depends on the adapter and on whether you re-prove from scratch or update incrementally.

• ▸Succinct adapter at N=8: the monolithic proof is smaller (the paper reports ~722 B vs ~5784 B modular) and verifies in one pass (~5.2 ms vs ~51.2 ms across eight passes); modular generation is ~3× more expensive. So for one-shot proving, monolithic wins.
• ▸Under the transparent adapter the size gap nearly vanishes (the paper reports ~32,357 B monolithic vs ~30,272 B modular at N=16).
• ▸What dominates is INCREMENTAL update: modular re-proves only the changed sub-chain at a constant ~0.039 ms, while monolithic re-proves everything at ~0.614 ms and rising linearly with length.

Reception throughput (the hot enforcement lane): the paper reports the gate accepting on the order of 5,146 notes/sec (median ~194 µs, p95 ~200 µs) and rejecting faster, around 16,070/sec (~62 µs, p95 ~64 µs). The asymmetry is by design: rejection short-circuits at the cheap authority/topic checks before signature verification. Dispersion is tight in both directions, which is what an enforcement path wants.

On-chain cost, stated precisely because it is easy to misreport: verification of a delegation proof costs roughly 2.10×10⁵ gas MEASURED UNDER A LOCAL VIRTUAL MACHINE, not on the public network. The public-network interaction was a read-only call that consumes and reports NO gas; a separate ~3.9×10⁵ gas figure recorded on the network is CONTRACT CREATION, not verification. The deployed verifier evaluates a single fixed-cost equation independent of chain depth — executing the circuit itself on-chain would be infeasible at ~9,151 constraints per hop.

On the same host and harness, the paper measures its own cleartext chain validation against two widely used alternatives: one that signs append-only blocks with public keys and evaluates a policy language, and one that chains message authentication codes. It compares chain-validation to chain-validation — not its ten-check reception gate — because those are different objects.

• ▸Its own lane is the SLOWEST and LARGEST of the three: the paper reports it running a few times (roughly 2–3×) slower than the public-key-block alternative and on the order of 15–28× slower than the MAC-based one, with credentials about twice the size of the first and an order of magnitude larger than the second, growing linearly with depth.
• ▸The gap is structural, not incidental: its lane does one independent public-key signature per hop and serializes in a canonical textual encoding; the block alternative verifies an aggregated signature over a compact binary encoding; the MAC alternative does symmetric authentication only, with no public-key operation.
• ▸The decisive caveat: the MAC verifier must hold the root secret, so every party able to CHECK is able to MINT — which forecloses non-repudiation to a third party. That is the guarantee the extra cost buys.

The conclusion the paper draws is explicitly NOT that its lane should be faster. It is that a deployment needing neither disclosure control nor cross-organizational verification should use one of the cheaper alternatives, and the measured cost is simply what the additional guarantees are priced at. “Validating a delegation” denotes three different computations here, so the table is read as three costs for three semantics, not one cost measured three ways.

The paper states the limits of its own numbers, and this candour is the part worth imitating.

• ▸Predicate fragments: only the transparent and pairing-based lanes prove the authorization check WITH signatures. The hash-based and folding adapters prove a fragment (attenuation + linkage) and their figures must be read with that qualification — the paper does not claim they prove the full check.
• ▸Execution-environment confound: the pairing-based lanes are measured in-process, while the two native-binary adapters run as separate processes and pay start-up, so cross-family verification ratios are confounded and are a harness limitation, not a property of the systems.
• ▸End-to-end mapping confound: the mapping from a persisted chain through the circuit retains prototype elements; every reported circuit number is attributable to a specific circuit and is not aggregated across them.
• ▸Dispersion: most lanes carry p50 and p95, but the two native-binary adapters emit a single value, so their figures are indicative rather than characterized; 50 samples is adequate for the spread observed but not for tails beyond p95.
• ▸Instrument contamination: the replay check performs a store write per accepted message on the measured path, which at realistic rates dominates before the cryptography does — so a naive report of the reception rate would substantially measure storage latency, and the paper flags this rather than hiding it.
• ▸Synthetic workloads: messaging and transparency measurements are deterministic and run without a live system; they characterize cryptographic and structural cost, not the behaviour of a deployed multi-organization population.

Expected direction: The ~2× from an idle machine is expected and legitimate-ish but must be labelled: the paper measured under ~2.1× load sensitivity on a non-idle host and reports figures as pessimistic by ~2×, so a quiesced machine naturally does better — it is not a new result, just different conditions. Disabling the store write is a measurement artefact: the paper flags “instrument contamination,” noting the per-message replay-store write dominates the measured path at realistic rates, so removing it measures the cryptography alone, not the reception gate, and also removes the at-most-once guarantee — so the higher number does not describe a correct system.

Why does the paper insist every performance figure be read as pessimistic?

Because all figures were taken on a deliberately non-idle host under realistic concurrent load, with a measured load-sensitivity factor of about 2.1; a quiesced machine would do better, so the reported numbers are the pessimistic end on purpose.

Why is scope-expressiveness cost described as non-monotone, and what does cost actually follow?

Because the pattern/wildcard level is the least expressive of the three yet the most expensive (reported ~59× the flat level); cost follows the decision procedure (pattern subsumption is pairwise structural comparison) rather than the expressiveness ordering.

State the on-chain cost precisely, distinguishing the three gas-related facts.

The paper reports ~2.10×10⁵ gas for verification measured under a LOCAL virtual machine; the public-network call was read-only and consumed/reported NO gas; and a separate ~3.9×10⁵ gas figure on the network is contract CREATION, not verification.

When does modular composition beat monolithic, per the measurements?

When chains change and you re-prove incrementally: modular re-proves only the changed sub-chain at a constant ~0.039 ms while monolithic re-proves everything at ~0.614 ms and rising; for one-shot proving, monolithic is smaller and faster.