Evaluation, in depth

Read the experimental evidence at full resolution: the exact setup, each measured result and what it does and does not establish, the containment metric and its subtle interpretation, the projected (untested) anomaly property, and an honest account of the ablations and baselines the paper does not provide.

Track

Research

Module

5 · Research case studies

Prerequisites

Architecture & algorithms, in depth

Before any number, fix the scale of the claim. The evaluation is one session, one context, one document type, a few dozen documents. That is enough to demonstrate that the mechanism behaves as designed; it is not enough to establish generalization, robustness, or advantage over a simpler baseline. Every figure below should be read with "on this corpus, in this session" attached.

The reported configuration, paraphrased and kept vendor-neutral:

the paper reports — setup
corpus:      ~31 documents, a single legal/tax document type, one production session
             (client identity anonymized by the authors)
embedding:   a 3072-dimensional text-embedding model
sensor:      a commercial LLM used for structured extraction
verifier:    three symbolic checks (a freshness check, a source-triangulation check,
             a contradiction check) with fixed similarity thresholds
calibrator:  k = 5 neighbors, alpha = 0.05, K = 8 document-type experts,
             n_min = 10, N = 100

Single session

All documents processed in one run. There is no across-session or temporal-drift evaluation, so claims about long-run learning dynamics are not tested.

Homogeneous corpus

All documents are the same type. This is what makes the cross-type anomaly claim impossible to test here — there is no second type present.

Production (not held-out)

The data is real operational traffic, which is realistic, but there is no held-out validation split and no ground-truth correct-confidence labels.

MEASURED: the paper reports that at the designed document count the layer transitions from cold-start (pass-through) to active expert mode exactly as specified. This is primarily a correctness check that the state machine fires at n_min, not a performance claim.

the paper reports — mode at milestones
n = 0    mode = cold_start   (pass-through; no calibration)
n = 10   mode = knn           (experts activate, as designed)
n = 31   mode = knn           (steady operation)

What it establishes: the activation logic works. What it does NOT establish: that activating produced better decisions than staying pass-through — there is no comparison of outcomes with vs. without the layer.

MEASURED: the escalation threshold decreased along its specified schedule as documents accumulated, matching the formula exactly (part 4 derived these values).

the paper reports — threshold vs. the specified schedule
n = 0    tau = 0.950   (maximum oversight)
n = 10   tau = 0.935
n = 31   tau = 0.903   (most routine documents handled autonomously)

cross-check with tau(n) = 0.95 - 0.15*min(n,100)/100  -> identical.

This is a consistency result: the implementation matches the model. It demonstrates the oversight rate relaxing with evidence, which is the intended behavior. It does NOT by itself show the relaxation was warranted — that would require knowing the true error rate on the autonomously-handled documents, which is not reported.

MEASURED: the paper reports that all documents in the corpus produced embeddings correctly stored at full dimensionality (verified to be 3072-length for every row), confirming the persistence-and-reconstruction path works. This underwrites the claim that expert memories survive restarts and are rebuilt in linear time.

The paper defines an Epistemic Containment Rate (ECR) over beliefs generated under adversarial or out-of-distribution conditions: the fraction of such invalid beliefs the verifier gate intercepts before they reach the planner. A correctly functioning gate should give ECR = 1 (everything bad is caught).

the paper reports — ECR definition and result
ECR = |{ b in B_inv : verifier(b) != VALID }| / |B_inv|
      where B_inv = beliefs generated under adversarial / OOD conditions
      target (healthy gate): ECR = 1

reported: ECR = 0.0 across all 31 in-distribution documents

Here is the subtlety that is easy to misread. The aspirational "ECR = 1 means all bad beliefs intercepted" applies to adversarial/OOD inputs — and those were NOT present in this corpus. The reported ECR = 0.0 is on in-distribution documents, where the meaningful reading is: no valid belief was wrongly blocked; calibration introduced no false negatives on clean data. So the number is evidence of "did not break the happy path," not evidence of "catches attacks." The same metric name is used in two regimes, and only the clean-data regime was measured.

PROJECTED (not measured): the property the paper is most excited about is that an out-of-type document escalates even when the sensor is confident. The argument is geometric and we can reproduce it, but it was never run against a real second type because the corpus had only one.

the paper reports — projected cross-type behavior (synthetic arithmetic)
context has only type-A memories (e.g. all "acta" documents).
an out-of-type document (e.g. a "factura") arrives; sensor is confident: c_llm = 0.95.

  the out-of-type embedding has LOW cosine similarity to every type-A neighbor
  -> the type-A expert's weighted average is pulled well below 0.90
  -> c_cal << c_llm, landing below tau(31) = 0.903
  -> escalate to a human, despite the confident sensor

The reasoning is sound as a hypothesis: distance in embedding space becomes low calibrated confidence becomes escalation. But "a tax invoice would score low against invoice-less memory" is a prediction about geometry, not an observation. Until a heterogeneous corpus is run, the cross-type anomaly claim is a well-motivated expectation, and should be cited that way.

An honest evaluation lesson has to name the experiments that were not done. The paper reports no ablations and no baseline comparison. We do NOT supply numbers for them — none exist — but we can state precisely what each missing experiment would have isolated, so you can weigh the evidence accordingly.

• ▸MISSING baseline — calibration vs. none: compare decision outcomes with the layer active vs. pure pass-through on the same stream. Without it, we cannot attribute any benefit to calibration.
• ▸MISSING baseline — MoE vs. a single calibrator: the paper argues a single per-context calibrator would fail across document types, but with one type in the corpus this argument is not tested; a heterogeneous corpus could.
• ▸MISSING ablation — k (neighbors): k = 5 is asserted; sensitivity to k is unmeasured.
• ▸MISSING ablation — alpha (learning rate) and the reward magnitudes: the curriculum's dependence on the 0.05 step and the +1.0 / 0.80 reward gap is unexplored.
• ▸MISSING ablation — threshold schedule: tau_max, tau_min, N are fixed constants; their effect on the oversight/error trade-off is not swept.
• ▸MISSING — statistical power: a few dozen documents in one session yields no confidence intervals, no variance, no significance testing.

Where would this system plausibly underperform, reasoning from the mechanism (these are analyses, not reported failures)?

• ▸Cold-start window: for the first n_min documents the layer is pass-through, so it offers zero anomaly protection exactly when a context is newest and least characterized.
• ▸Sparse experts: an expert with fewer than three memories abstains, so rare-but-legitimate document types get no specialist and fall back to the raw sensor confidence.
• ▸Homogeneous history breeds overconfidence relaxation: a long run of one easy type steadily lowers tau, which is good for throughput but means a later subtle in-type error faces a looser threshold.
• ▸Reward bias: with only operational feedback and no ground truth, systematic reviewer bias or non-random escalation is learned as signal.
• ▸Single session hides drift: nothing tests what happens when the input distribution shifts slowly over weeks — precisely the regime the layer is supposed to help with.

None of these are disqualifying; they are the natural boundary of what a single small-corpus demonstration can show. The right summary is that the paper proves a structural property rigorously and demonstrates the mechanism narrowly, while the performance and robustness questions remain open.

Expected direction: The misreading conflates two regimes of the same metric. ECR = 1 (not 0) is the target for adversarial/OOD inputs, and those were not present in the corpus. The reported ECR = 0.0 is on in-distribution documents and means no valid belief was wrongly blocked — i.e., calibration introduced no false negatives on clean data. A correct rewrite: "On in-distribution documents the paper reports ECR = 0.0, meaning calibration did not newly block any valid belief; the system's containment of adversarial/OOD inputs was not measured and is left to future work."

What is the scale of the evaluation, and why does it bound the claims?

A single production session over a small homogeneous corpus of one document type (reported as ~31 documents). It can demonstrate the mechanism but cannot establish generalization, robustness, or advantage over a baseline.

Why can the cross-type anomaly property not be a measured result here?

Because the corpus contained only one document type, so there was no out-of-type document to escalate; the behavior is projected from k-NN geometry, not observed.

Name two experiments the paper does not provide and what each would isolate.

Examples: a pass-through baseline (would isolate whether calibration improves outcomes) and a k / alpha / threshold-schedule ablation (would isolate sensitivity to the key hyperparameters). No ablations or baselines are reported.