Memory and retrieval in agents

Distinguish short-term/working memory from long-term episodic and semantic memory, decide what should persist and what must never become a belief, treat memory as an attack surface (poisoning, unbounded growth), and design memory with provenance so that retrieved content is reviewed rather than silently trusted.

Track

Engineering

Module

10 · Agentic architectures

Prerequisites

The LLM as a component: ReAct

Adding a vector store to an assistant does not make it an agent. What makes memory matter is the loop: what the agent remembers conditions a decision, the decision has an effect, and the effect comes back as something new to remember. Once memory steers actions with consequences, the only honest question to ask of it is not "what did we see?" but "does what we remember justify what we are about to do?".

That reframing is the whole lesson. A pile of retrieved passages is a record: it grows, it is good for auditing, and it leaves interpretation to whoever reads it. A decision needs something else — a view that says, for each thing that matters, what is believed right now and why. The common mistake is to keep only the record and ask the language model to act as the view on every query. That turns belief revision into something stochastic, implicit and unrepeatable: the same store can yield a different decision twice, for reasons no one can reconstruct.

The agent literature borrows a vocabulary worth keeping straight. Working (short-term) memory is the scratch space of the current task — the context window, intermediate results, the plan in flight. Long-term memory splits into episodic (what happened, in order), semantic (consolidated facts and relationships) and procedural (skills and learned routines). Consolidation is the act of promoting the episodic record into semantic knowledge, because the raw record grows without bound.

Working / short-term memory

The volatile state of the current episode: context, partial results, the current goal. Cheap to read, lost (or summarized) when the episode ends.

Episodic memory

An append-only record of observations in temporal order. It only grows; its value is audit and reconstruction, never a fast answer.

Semantic memory

The consolidated "what is true now" view — one value per key — produced from the episodic record. Cheap to read because the review was done at write time.

Record vs. belief

A record stores what was seen and defers interpretation. A belief is a view: the current value for a key and the justification behind it. An agent needs both; conflating them is the root failure.

This maps onto a much older engineering idea. A deliberative agent's memory has always been a belief base, and the discipline of revising beliefs rationally when new information contradicts the old is decades old — truth-maintenance systems recorded, for each belief, the justifications that held it up, and retracted in cascade whatever lost its support. Modern data-provenance models are the same bookkeeping under a new name. The lesson is simple: if you store why you believe something, you can stop believing it when the why disappears.

The mechanism is a typed memory entry plus a deterministic rule that turns the record into a belief. Authority is relative to the key: the order system is authoritative for a delivery date but not for the refund policy; a customer is authoritative for their own contact preference but not for whether a supervisor granted them an exception. Fixing the authority function is a domain decision, made by engineers — never inferred by the model.

synthetic — typed memory entry and the belief rule (deterministic, outside the model)
entry = {
  id,              # unique handle; used for cascade retraction
  key,             # (entity, attribute), e.g. ("order-91", "delivery_date")
  value,
  source,          # who asserted it: a system of record, a user, the agent itself
  authority,       # A(source, key): is this source authoritative FOR this key?
  observed_at,     # when it entered the store
  type,            # observation | claim | inference | summary
  valid_from, valid_to,
  justification,   # set of observation ids this entry rests on (summaries/inferences)
}

belief(key, t):                       # the rule that decides what is believed
  candidates = { e in store :
      e.key == key
      and e.type in { observation, summary }      # NOT claims, NOT inferences
      and e.valid_from <= t < e.valid_to }
  if candidates is empty: return declared_default(key)   # or re-perceive
  return argmax(candidates, by = (authority, observed_at))  # authority first, then recency

Two properties fall out by construction. First, no third-party claim and no inference of the agent's own — however numerous or recent — can change a belief, because neither type is ever a candidate. Second, if every summary keeps in its justification the observations it came from, retracting an observation deletes exactly the summaries that depended on it. The honest limit is also visible in the rule: the belief is the highest-authority valid observation, so if that source is wrong, the belief is wrong. Provenance does not make memory true.

Retrieval-augmented generation is usually wired as "fetch passages, drop them in the context, let the model sort it out". That quietly delegates belief revision to the model, which then resolves — in silence, and differently each time — the conflict between the old and new version of a policy, between the system of record and what the customer said, and between a fact and a note the agent wrote yesterday. Long-context position effects make it worse. The fix is to treat each retrieved passage as a candidate observation that must be typed and reviewed before it reaches the decision.

1. 1Retrieve: the query returns passages, each with its source and timestamp — not bare text.
2. 2Type: tag each passage by the authority of its source over the key it speaks to (observation, or third-party claim).
3. 3Review: run the same deterministic belief rule over the typed candidates that you run over any other observation.
4. 4Decide: the policy reads one belief per attribute, not eight passages, and the model never arbitrates the conflict.
5. 5Keep the model at the edge: use it to extract a key and value from a document, to notice that two entries are about the same thing, or to draft — never to decide what is believed.

The write rule is easy to state and easy to get wrong. What comes from a source is stored with that source. What a third party says is stored as a claim by that party — "the customer asserts an exception", recorded as the fact that they asserted it, not as the exception being real. What the agent concludes is stored as an inference, with its justification, and it justifies nothing downstream. Claims are not discarded: they trigger verification against an authoritative source and remain for audit.

The failure mode this prevents is the echo. If the agent writes "customer is entitled to an exception" with no type, a later retrieval reads it back as evidence of what it itself concluded. Worse, a flat store where each read picks a passage and writes a note using it behaves like a reinforcing urn: the proportion of true-vs-false passages freezes wherever the first few reads happen to push it, and a new true value arriving into a store full of old notes can be outvoted forever. Typing own-writes as inferences — which are never candidates — makes that dynamic impossible by construction.

• ▸Persist: observations from authoritative sources (with source, authority, time, validity), and summaries that cite the observations they rest on.
• ▸Persist but never promote to belief: third-party claims (stored as "X asserts φ") and the agent's own inferences (stored with justification).
• ▸Do not persist as fact: anything the model generated that is not backed by a cited observation — a summary that cannot name its sources is a laundered claim.
• ▸Forget cheaply, but only the record: least-used eviction is a capacity policy on the episodic store, not on the beliefs that justify current decisions.

Whatever is written to memory persists and is retrieved in future sessions, which makes it a durable attack surface rather than a one-shot one. Two public classes of attack are worth naming: memory poisoning, where an attacker gets malicious content written so it is later retrieved as trusted context, and query-only memory injection, where crafted inputs alone steer what the store ends up believing. A shared memory across several agents is the amplified version — one poisoned write is read by all (see the multi-agent threat landscape). Unbounded growth is the quieter attack surface: a store that only grows eventually drowns the signal and forces lossy consolidation, which is exactly where laundering sneaks in.

Typing by source is the precondition for every defense here, and consolidation is where it is tested. If a language model writes the summary, that summary is a generation and can assert what the sources do not. The architecture forces it to cite the observation ids each value came from and checks, in code, that the citation exists, is an observation, and is the one the belief rule would have picked.

synthetic — consolidation with a provenance validator, and cascade retraction
consolidate(key):
  draft   = model.summarize(observations_for(key))   # the model MAY draft the text
  cited   = draft.cited_ids                           # ids it claims to rest on
  chosen  = belief(key, now)                          # what review would actually pick
  if not (cited are observations) or (cited != chosen):
      reject(draft)                                   # refuse to launder a claim into a fact
      return
  store.add(type = "summary",
            value = draft.value,
            authority = chosen.authority,
            valid_from = chosen.valid_from,
            justification = chosen.justification)      # inherits provenance -> retractable

retract(id):
  # withdraw an observation AND every summary that only it justified
  store.remove_all(e where e.id == id or id in e.justification)
  # a summary stored WITHOUT justification cannot be retracted when its source is withdrawn:
  # no procedure, given only the store, can tell whether it should now go

• ▸Provenance does not make memory true. It makes each belief as good as its best source, and makes beliefs auditable and retractable — nothing more. When an authoritative source is itself wrong, you need corroboration from independent sources, not better bookkeeping.
• ▸Extraction is still stochastic. Turning a document into typed entries needs a model whose error enters the store before the deterministic review ever runs; the review cannot fix a key or value that was misread.
• ▸Validity is a function of the action, not a constant of the memory. Whether to re-validate a belief before acting depends on how often the value changes and on the relative cost of asking versus acting on stale data — a cheap read with an expensive error should be re-validated more eagerly.
• ▸Three ways a belief stops holding: a newer observation supersedes it, its declared validity expires, or the world changed silently. The first two the review handles if entries carry time and validity; the third can only be caught by re-perceiving.
• ▸Memory benchmarks mostly measure whether something is recalled, not whether what is recalled justifies the action. A store can pass recall tests and still feed a confident, well-sourced, wrong decision into an irreversible effect.

Expected direction: In the flat store the customer message becomes just another passage that can win a read; the agent's own "window 30" notes outnumber the single new policy document, so an echo keeps the stale window alive (a reinforcing-urn dynamic). In the typed design: the customer message is stored as a third-party CLAIM ("customer asserts an exception"), which never becomes a belief but triggers a check against the authoritative supervisor record; the window is the belief from the policy document valid on the request date; and annulling the exception retracts that observation by id, which cascades to delete every summary whose justification cited it. The learner should note that provenance makes the exception retractable but does not, by itself, make the authoritative source correct.

Why is it a mistake to store retrieved passages and let the model act as the belief view on each query?

Because belief revision then becomes stochastic, implicit and unrepeatable — the model silently resolves conflicts between old and new values, so the same store can yield different decisions and no one can reconstruct why. The deciding rule should be explicit and deterministic; the model stays at the extraction/phrasing edge.

What is the difference between a third-party claim, an observation, and an inference in a typed memory, and which can change a belief?

An observation comes from a source with authority over the key and CAN be a belief candidate. A third-party claim records that someone asserted something ("X asserts φ") and triggers verification but never becomes a belief. An inference is the agent's own conclusion, stored with justification, that justifies nothing downstream. Only observations (and provenance-backed summaries) can change a belief.

Why can a summary stored without justification not be retracted when one of its sources is withdrawn?

Because nothing in the store links it to the observations it came from. Two stores — one where the summary rests on the withdrawn observation and one where it rests on a different observation of the same value — are identical after a no-provenance consolidation, so no procedure using only the store can decide whether the summary should go. Keeping the cited ids makes cascade retraction exact.

In what sense is memory an attack surface, and what is the precondition for defending it?

What is written persists and is retrieved as trusted context in future sessions, so poisoned or injected writes become durable, re-readable influence — amplified when memory is shared across agents. The precondition for every defense is typing by source: untrusted-origin content is stored as a claim, not as a fact, so it cannot silently become a belief.