pi-canon Part 1 · The Design, Measured Whole
One canonical article per governed asset, at an address you can compute. An append-only journal underneath it keeps the first version the agent records. Recall arrives on its own, when a tool call touches the thing the knowledge is about.
Scroll to explore
The Lineage
You have probably already seen the LLM wiki
A folder of markdown articles the agent writes and rewrites, linked to each other, with no schema declared in advance. It is a good idea and it is not a new one. Many file-based agent memory systems converge on this shape, and what they get right is worth keeping: plain files, no database, and knowledge free to grow in whatever shape the project actually has.
That freedom is the celebrated part. It is also where these stores fail, and they fail in two specific directions rather than degrading vaguely.
The first is scatter. Nothing marks which article is the article about a topic, so an agent that cannot find it writes another one. There is now a note on the vendor feed, a second on feed pagination, and a third on the sync job, all describing the same constraint from three angles, none of them wrong, and no way to tell which to trust when they disagree. Retrieval finds all three. The agent reads whichever ranks highest, which is not reliably the one that is current.
The second is log drift. A store an agent writes to during work fills with events, because work is made of events: what was tried, what failed, what got fixed. Current truth ends up buried under a log of how it came to be true, and the page that should say what the rule is says what happened on the fourteenth instead. Neither failure errors, neither announces itself, and both compound every time the store is used.
Neither is a storage failure: in both, the knowledge is present. Scatter is an addressing failure, leaving precedence undefined across copies. Log drift is a mutability failure as well, putting event history and current reference knowledge in one page that a later rewrite can edit out from under both.
pi-canon keeps that store and keeps its freedom. It adds three things to it. A journal, because agents log; given only a reference store they write events into it, and the journal is where that impulse is sent instead. A spine, so a topic's article has an address computed from the asset rather than searched for. And surfacing, so the article arrives on its own when work touches what it governs, because stored knowledge has no effect unless it reaches the session that needs it.
None of that is a new storage engine. It is still markdown in a folder, committed with the repository, and git supplies the history. What changes is where an article lives, what is allowed to be rewritten, and whether anyone has to think to ask for it.
It is worth being exact about how much of that the runtime actually holds, because the defensible claim is narrower than the ambitious one and it is the narrower claim this page makes. Three things are constrained by the code: a journal file cannot be rewritten once written, a governed path resolves to one article once that article exists, and a recognized touch on a governed asset can push that article's capsule into the session. Everything on the write side remains the agent's judgment. It decides that an event is worth journaling, which asset should govern a piece of knowledge, and how faithfully to distill the source into the article. The design constrains three failure points; it does not automate the writing.
What the page does not claim is that the evaluation further down attributes the result to any one of the three. That study is package-level by construction, and which addition carries how much of the gain is still open.
Yes, another LLM wiki, but this one has a spine
An illustrative repository as a graph: thirty-three articles, twenty journal entries, and forty files and directories. Twenty-seven articles sit at a computed address, tethered to the asset there; six match no asset and are drawn no differently beyond a dashed outline. Hue is the semantic cluster, across nine clusters, and each cluster is pulled into its own region. client.py and summary.py tether one hop up to the articles governing their directories, and the tools subtree attaches to nothing because nothing governs it. Parent articles route to their children, and journal entries hang beneath the article each was distilled into.
The store the section above describes, drawn the way these stores are usually drawn: markdown articles in a folder, linked to each other, in whatever shape the project turned out to have. Drag a node, drag the ground to pan, or expand it to the whole viewport. What is different does not show up in the shape at all; it shows up in the labels. Almost every article here sits at an address that was computed rather than chosen, and the square tethered under each disc is the asset it was named for: the article at src/feed/sync governs src/feed/sync.py, and nobody maintained that correspondence. Three consequences are visible here and nowhere else on this page. Coverage: every article sits directly above the asset at its address, while client.py and summary.py, having no article of their own, tether one hop up instead, and the tools subtree floats off to the side attached to nothing, because an ungoverned subtree has nothing to attach to. Hierarchy: a parent article is a summary plus routes, and those routes are ordinary edges rather than a separate mechanism. And the six articles that match no asset at all sit in the same clusters as everything else, because free knowledge is not a special case here; the only thing distinguishing them anywhere is that none has a tether down to a file. The rings are the second tier, the append-only journal, one file per event, filed under the article it was distilled into. Colour is the semantic cluster an article belongs to, sampled from one perceptual ramp rather than picked per category, and the layout pulls each cluster into its own region, so a band of the palette and an area of the map mean the same thing. Drop the repository tier with the buttons above and what is left is the canon by itself, which is exactly the graph you already know, except that most of its addresses were never chosen by anyone. Illustrative repository; the tiers, addressing and article shapes are the ones the package writes.
The Spine
An address you can compute, and a rule you can follow without stopping to think
The address of an asset is the asset's path with its file extension dropped. The article at articles/src/feed/sync.md governs src/feed/sync.py. Identity mapping needs no configuration, which is what makes the spine a convention rather than a mode, and resolution walks up to the nearest existing ancestor, so the rule is at most one canonical home per asset rather than one article per file. Renames are a file move. The article travels with its asset.
What predictable addressing buys is a doctrine cheap enough to actually follow: resolve and read the governing article before working on an asset, and revise it after. Of the four things anyone does to a store, the weight here sits on reading, updating and deleting; creating an article is the uncommon act, reserved for an asset that has no governing article yet. An address derivable from the asset itself is what makes that read habitual rather than aspirational, and it is why no part of the package searches.
The spine is optional, and that is not a concession. An article matching no asset is simply free knowledge, no flag distinguishes it, and it links with spine-attached articles in both directions. The spine guarantees an address for the assets a project already has; it does not confine the store to them. A decision that belongs to no file stays in the store as an ordinary article. The tradeoff is real and worth stating: today's surfacing is deterministic and asset-scoped, so an off-spine article is found by a link or a read rather than pushed on a touch.
Growth has a shape too. An article past its size band splits at asset boundaries first; past that it goes hierarchical, and the article that was a leaf becomes the parent of its own subtree, a summary of the subject on top and routes to the children carrying the detail beneath. A governing article for a subtree is therefore two things at once, which is what lets one article answer for a subtree without pretending to contain it.
Pick any file, and the address is already decided
An illustrative repository tree where most paths carry their own article. Selecting a path resolves it to a governing article and shows the one-line capsule that surfaces automatically on a touch, with the full article body available behind it. Parent articles are a summary plus routes to their children. src/feed/client.py and src/report/summary.py resolve one hop up to src/feed and src/report. tools/cron_pulse.py resolves to nothing, because the tools subtree is ungoverned.
The spine is the whole addressing scheme, and it is not a component: no per-asset mapping table is ever written, and a rename is a file move. Someone still chooses which asset becomes a governing unit when they create its article; after that choice, every descendant resolves to it without anyone maintaining the link. Click any row. Most paths here have an article of their own; client.py and summary.py do not, and walk one hop up to the article governing their directory, which is what lets one article go on governing a subtree without anyone maintaining a list of what it covers. tools/cron_pulse.py walks to nothing at all, because an ungoverned subtree has no capsule to offer. Two things are worth separating as you click. The capsule is the one dense line that arrives on its own when a tool call touches the asset, unasked; the article is everything behind it, and the agent reads it only if it decides to. Open a parent such as src/feed and the shape lint asks for is visible: a summary of the subtree on top, routes to the children carrying the detail beneath. Illustrative visualization; the capsule, body and router shapes are the ones the package writes.
The Model
Two tiers with opposite mutability disciplines
The canon is mutable and holds the current best understanding. One article per governed asset, rewritten in place as understanding changes, so an asset resolves to one page to read and one page to correct. The journal beneath it is append-only, one file per event, and the agent is instructed to preserve source wording, names, and exact numbers when it writes there.
The split is doing real work. A mutable page is the right shape for a rule, because a rule that changed should not leave its old self lying around to be found later. An immutable log is the right shape for an event, because an event that happened cannot stop having happened, and because the exact phrasing is often the part that matters: the error string, the vendor's own words on the call, the number someone quoted.
The obvious objection to adding a journal is that it doubles the store and recreates the same bloat one directory lower, so the intake rules matter as much as the split does. The journal is an outlet, not a second recall payload. The canon is read first and is the ground truth; the journal is the sequence that led to it. Reading an article lists the filenames of the newest journal entries beneath it and nothing more, so history is on offer and never loaded by default, and surfacing never injects a journal body or a journal summary.
This maps onto a distinction cognitive psychology drew decades ago, and the mapping is a description rather than a derivation. Episodic and semantic memory are the two halves of long-term declarative memory: the journal is the episodic half, temporally indexed and immutable, and the canon is the semantic half, the current understanding with the history stripped out. The mapping is a description of what the two tiers already are, not an argument that the design is brain-shaped.
Distillation is where it goes wrong, and the run behind this page shows exactly how. An article is a distillation of what the journal recorded, and distillation loses things. When it loses the operative clause of a constraint, the store is left holding a page that reads as authoritative and no longer says the thing that mattered.
What the journal keeps, and what the article can lose
A fact arriving in a session and splitting into two tiers. The journal entry preserves the source wording verbatim and is append-only. The article distills it and is rewritten in place by later sessions. A later rewrite drops the operative clause of the rule, leaving a page that still reads as authoritative; the laundering guard quotes the dropped language back at the write that removed it, and the journal still holds the original.
The same constraint, run through both stores. Start on one mutable page, which is the shape the opening section described: the rule goes onto the page about the report, later sessions add what they did to the same page because there is nowhere else for events to go, and a later session condenses the result. That rewrite is not careless. The page had become a log and tidying it is the right instinct; what the agent cannot see is which line was a constraint and which was an event note. Switch to journal and article and the same five beats run through two tiers written by two separate calls, which is agent work rather than an automatic fork. The article is rewritten just as freely, and it ends up stating the same wrong thing. Two differences survive. The journal entry underneath still holds the vendor's own phrasing, and the laundering guard quotes the dropped line back at the write that dropped it, cut off at 160 characters, which in this case lands mid-word and before the operative detail. Neither undoes the rewrite, and the article is still wrong at the end of both sequences. What changes is not whether the sentence survives anywhere, since this store is committed with the repository and git keeps the old revision either way. It is what a later session meets. On one page the sentence is a deleted line in a revision an agent has to know existed before it can go looking. In the two-tier store it is a current file with its own name and date, indexed under the article, reachable without a commit to diff against. The measured section below counts how often the write desk lost material even so. Illustrative visualization; each quoted article, journal line and lint advisory in the two-tier sequence is verbatim from the run-2 artifacts, drawn from two repetitions of the consumer-contract chain rather than from one cell.
Recall
The agent does not have to know there is something to ask
Retrieval systems answer questions. That is their strength and it is also their blind spot: a search only runs when something thinks to run it, and the expensive failures in project work are precisely the ones where nobody knew there was a question. An agent about to prettify a number in a report file does not stop to search for whether a downstream parser is fragile. It has no reason to. It does not know the parser exists.
So recall here is push rather than pull. When a tool call is detected touching a governed asset, that asset's article contributes a one-line capsule to the session, at most once per article per session, under a hard budget. Nothing is searched, and the agent never had to suspect anything.
The discipline around that is mostly about not being annoying. A capsule that arrives when the agent has already gone and read the article itself is noise, so reading an article through the package withdraws the line staged for it: pull preempts push. That is the article, not the asset. Reading src/report/writer.py does not cancel anything, because reading a source file is not reading what is known about it, and the capsule may hold exactly the constraint the file does not contain. Past the session budget an article still surfaces, but only as a pointer saying it exists and should be read, rather than spending tokens on its contents. And an article counts as seen only when its line actually rode out in a flushed message, so a nudge that got overtaken costs nothing.
The caveat belongs in the same breath: detection is best effort. Surfacing fires on tool calls the runtime recognizes as touching a governed path, and an agent that reaches an asset by a route the runtime does not recognize gets nothing. The mechanism is a push with a hit rate, not a guarantee.
A session, and the capsules that reach it
One session of three turns. A tool call stages the article governing whatever it touches and delivers nothing; the turn boundary flushes everything staged as one message, because the steering queue drains one message per provider round trip. An article surfaces at most once per session, capsule text is capped at 4,000 characters for the session, and past that an article still surfaces as an uncharged pointer. Reading an article through the package withdraws its staged capsule and marks it seen. In the run-2 artifacts 19 capsules were staged, 13 were withdrawn by a pi_canon read of the article, and 6 were flushed.
One session, three turns, drawn from the rules the package actually applies. A tool call stages the article governing what it touches and sends nothing; the turn boundary is where staged lines become a single message, because the steering queue drains one message per provider round trip and a message per tool call would buy every nudge its own model call. Select any event for the three gates in order. Whether a path was detected matters first, and it is the weakest link: ls src/feed yields a path, cat package.json yields none, because a bare filename inside a longer string is not a candidate. Whether an article resolves is separate, and deterministic once a path is in hand, though it can still walk to the top and find nothing. Only then does anything get delivered. Watch the explicit article read withdraw a staged capsule and mark that article seen for the rest of the session, which is what makes every later touch of it silent. The budget control sets how much of the 4,000 character capsule allowance the session had already spent before this trace: run it forward and articles degrade to uncharged pointers rather than vanishing, and a short capsule can still land after a long one was refused. Below, the same funnel as measured: across the canon-family arms of run 2 the trace recorded 19 capsules staged, 13 withdrawn by a pi_canon read of the article, and 6 flushed. Illustrative session; the mechanics and constants are the package's, and the counts are measured from the run-2 trace.
The Seam
What the runtime holds, and what it only asks for
All three mechanisms are now on the table, which is the right moment to draw a line through them. Not the boundary in time that the stack figure below turns on, where a session ends and two tiers of memory die with it, but a line through responsibility: what the code will not let go wrong, against what it can only ask for. A reader who has got this far could reasonably conclude that the design removed a dependency on model behaviour. It did not. It moved the dependency to one side of a line and constrained the other side, and the two sides are worth naming separately, because the whole page rests on the left column and the open problems all live in the right one.
Held by the runtime
- A journal entry is written with a create-exclusive write, so the package never rewrites or deletes one. The files stay ordinary Markdown, and any other tool can still edit them.
- Once a path is detected, it resolves to exactly one article, walking to the nearest ancestor that has one, or to nothing at all. Finding the path in a tool call is best effort; the walk that follows is not.
- An article surfaces at most once per session, and capsule text is capped at 4,000 characters for the whole session. The cap counts capsule bodies alone: the message header, the pointers that replace capsules past the cap, and the write-after reminder are all unbounded by it. The test also runs per capsule against what is left, so a short capsule can still land after a long one was refused.
- Reading an article through the package withdraws its staged capsule before the message goes out, so pull preempts push.
Asked of the agent
- Decide that an event is worth journaling at all.
- Choose which asset, or which ancestor, should govern a piece of knowledge.
- Write the article as well as the journal entry; the two are separate calls, not one automatic fork.
- Distill the source faithfully, and preserve the operative clause when revising later.
- Give the journal entry the right subject, since that is what files it under the article a later session will reach it from.
- Open the article when a capsule or a pointer says there is one. A line in the context is not a read.
- Then follow it, against a live prompt asking for something else. Nothing in the package makes a rule win that argument.
The laundering guard sits on that seam and does not close it: a write that drops a line matching a small constraint vocabulary gets the first 160 characters of that line quoted back at it, which is a prompt to the agent rather than a refusal by the runtime. The quote is a prefix, not a summary, and across the archived runs it stops mid-sentence more often than not. Nothing in the package can compel an agent to keep a line it has decided to cut.
That distinction is not a weakness to be managed quietly, because there is evidence about which column actually breaks, and it is not the one the design spends most of its effort on.
A development run traced every missed fact back to its first failure point, and the answer was lopsided enough to redirect the work: of fourteen misses, thirteen died at the write desk. Eight were never captured into any tier, five were captured and then overwritten by a later rewrite. None were sitting in the store unread.
Which is an uncomfortable result, because every one of those thirteen sits in the right-hand column. Surfacing worked. Addressing worked. What failed was the moment of writing down: a distillation that had already lost the operative clause, or a later session revising a page and taking the rule with it. The runtime guarantees held and the agent obligations did not, which is the same seam drawn above, arriving as a measurement rather than a caveat.
Where a fact actually dies
The fourteen recall misses from run 2's two canon-family arms, classified by first failure point: 8 never captured into any tier, 5 captured then overwritten by a later rewrite, 1 judge error, and 0 lost at retrieval or surfacing. Thirteen of fourteen are write-desk losses.
Every miss classified by where it first went wrong. Hover any row for the mechanism, including the two verbatim examples, each quoting the plant prompt or the store snapshot it came from: a plant-time paraphrase that turned docs claim 1000, from the prompt, into even when a larger limit is requested in the article, and a later rewrite that reduced drops lines containing commas to brittle. The row that matters most is the empty one. Not a single miss was caused by the store holding a fact that recall failed to reach, which is the failure a retrieval-shaped design would predict. The response was wording rather than mechanism: the journal verb now asks for the source as it happened, names and numbers included, and both orientation branches close with the doctrine sentence. On a repeat of the same chains canon moved from 10 of 18 fact-level verdicts to 17 of 18. Those chains were directly reused during development, so the right sentence is that the change coincided with the improvement, not that it caused it. Measured from the run-2 forensic pass.
All of it fits in one picture, which is worth drawing only now that the pieces in it have names. The line the whole design answers to is a boundary in time: two tiers of memory end when the session does, and two persist. Everything above is a companion package's concern and optional in both directions; everything below is this one's.
One memory system, four tiers, and the session boundary that makes it hard
A four-tier memory stack: short-term is the raw context window, working is the folded window served by pi-fold, episodic is the journal of events as they happened, and semantic is the canon of one article per asset. A session boundary separates the two upper tiers, which die with the session, from the two lower ones, which persist. Capture flows downward, the journal preserving source wording and the canon distilling it. Recall flows upward, a touch on a governed asset carrying that article's capsule back across the boundary into a later session's raw window. pi-canon owns the two persistent tiers and the surfacing lane; pi-fold owns the working tier and is optional in both directions.
The whole system at once, now that its parts have been operated. The two tiers above the dashed line live and die with the session; the two below it persist, and the capture and recall problem this package addresses is a consequence of that line. Capture runs down the right: the journal takes the wording as it arrived, the canon distills it into one addressed article whose address is the asset path. Recall runs back up the left, and where that arrow lands is the point of it. The capsule re-enters the raw window rather than the folded one, which is why pi-canon needs no companion to work. Turn pi-fold off and the working tier empties with nothing below it changing, which is what it means for two packages to compose without depending on each other. The mapping onto episodic and semantic memory describes what the two tiers already are; it is not a claim that the design is brain-shaped. Illustrative visualization.
The Evidence
Nineteen of twenty, against eight
Recall benchmarks ask questions and score answers, which measures memory on the assumption that somebody knew to ask. The expensive failures in project work are the other kind, so the primary metric here is not recall at all.
One boundary belongs before the number rather than after it. No evaluated arm is a search-driven LLM wiki. The comparison arms are a self-maintained convention file, a static doctrine file beside the canon, and a floor with no memory system, so the run supports a package result against those baselines on these chains. It does not test the lineage argument made at the top of this page. Nothing here shows that pi-canon beats a disciplined wiki, that a journal fixes log drift, that the spine fixes scatter, or that surfacing is the component causing traps to be avoided. Those remain the reasons the design is shaped this way, not findings.
A chain is a small fictional repository and a four-session story sharing one worktree. A plant session does ordinary work whose natural course surfaces a constraint, never phrased as an instruction to remember. A distractor session does something unrelated. Then a probe session gets a task whose obvious solution violates that constraint in a way that compiles, runs, passes the visible tests, and fails a grader the agent cannot see. A failed trap is a change that looks finished, passes review, and breaks something the team already knew about.
On the confirmatory run, canon passed 19 of 20 trap cells against 8 for the bare floor, with a self-maintained convention file at 18 and canon plus a static doctrine file at 16. The floor's failures are the informative part, because they repeat rather than scatter: it failed the consumer-contract chain in all five repetitions and the timezone chain in four of five, which is what an arm with nothing carrying the constraint forward looks like. It is a stronger floor than the name suggests, though: prior-session transcripts are written into its worktree before recall and its agent is on record finding and reading them, so it is a transcript baseline at recall time and a no-extension floor at probe time.
The generalization unit is the binding limit. Five hand-built chains, four eligible trap designs, each repeated five times; repetitions repeat the same traps, so a hundred cells is not a hundred independent observations, and no uncertainty interval is attached to any pooled count.
Every eligible probe of the confirmatory run
Eighty trap-eligible probe cells, four arms by four trap-carrying chains by five repetitions. canon passes 19 of 20, agents.md 18 of 20, canondoc 16 of 20 and bare 8 of 20. Switching to every intended check removes the cold-control conditioning entirely and scores all 110 checks per arm: canon 109, agents.md 107, canondoc 105, bare 85.
One square is one probe cell. Defining traps through the floor arm's cold failures conditions the metric on that arm's behaviour, which is the strongest objection to the number, so the conditioning is a switch here rather than a footnote. Turn it off and every intended check counts, whether or not a cold worker had already failed it: the chain with no trap endpoint rejoins, so each arm carries 25 cells rather than 20, and a cell survives only when every check in it passed. Counted as individual checks instead, the same run reads canon 109 of 110 and bare 85 of 110. The ordering survives either way. Hover any cell to see which check failed and which checks were trap-eligible in that chain. Measured from the sealed run artifacts.
The Fine Print
What the run does not establish, in the same breath as what it does
Four boundaries bound the result, and none of them is buried further down. The five chains are development-exposed, because the product changed in response to failures on these same chains and the confirmatory run reuses them. The result is package-level: nothing here attributes it to the journal, the spine, or surfacing separately. Eligibility is a property of the worker model rather than of the task, so every number built on it moves when the worker does. And the same person wrote the package, the chains, the traps, the graders and the plant-only declarations. How each of those was decided is laid out in full below the next figure, unfolded for anyone who wants to check it.
The secondary metric found nothing, and it is worth saying plainly. Plant-only recall clustered at 41, 40, 42 and 40 out of 45 across the four arms, with the floor level with the memory system. The recall instrument does not reliably distinguish the arms, and the ordering is not stable: one fact, flagged as paraphrase-sensitive before the freeze, carries 9 of the 17 misses, and removing that single item rearranges the ranking so canon lands level with the floor. The right reading is that every arm recalls planted facts at roughly the same high rate and this instrument cannot separate them. Where the arms do separate is the price of the answer: canon answers recall at 0.34x the floor's median recall tokens, and the pattern is mechanical, because both expensive arms carry large persistent text into the recall session. Lower token use did not always mean lower total cost, and canon is never the cheapest arm; canondoc is lowest on both median recall tokens and total chain cost.
Clustered on recall, separated on the bill
Every plant-only verdict as a cell: nine facts by five repetitions by four arms. Totals are agents.md 42 of 45, canon 41, canondoc 40 and bare 40, clustered within three facts; removing the pre-flagged item reorders them to agents.md 40 of 40, canondoc 38, canon and bare level at 37. Below, median recall session tokens: canondoc 13,991, canon 20,775, bare 61,006 and agents.md 64,568.
The upper panel explains why an instrument was set aside; the lower one is the finding, and it gets the room. Every judged verdict is drawn rather than summarised, because the summary is what made this look decisive: at cell level the misses concentrate visibly in one row, the item flagged as paraphrase-sensitive before the freeze. Switch to the sensitivity view to strike that row out and watch the ranking rearrange on the removal of a single judged fact, which is what it means to say an instrument cannot separate its arms. The frozen totals stay printed beside the recomputed ones, because a sensitivity view is not a second result. Below is where the arms do separate. Each point is one recall session, laid out in its own chain rather than as one cloud, so the dependence is visible: a chain's five points are five repetitions of one workload, not five independent draws. The distribution is consistent with how the arms are built, since agents.md and bare both re-read large persistent text at recall time; the figure shows the association rather than demonstrating the mechanism. Hover any median tick for that arm's recall cost and total chain cost. Every dollar figure is metered worker-session cost; the judge's calls are archived but never metered, so no cost figure on this page includes them. Measured from the sealed run artifacts.
The instrument in full: how eligibility was decided, and what one cell looks like
Everything below is benchmark methodology rather than product. It is here so a skeptical reader can check the claims above without leaving the page, and it is folded away because it explains how the study was built rather than what the package does. The two figures also appear in the paper.
A trap only tests memory if a model without memory actually falls into it, so eligibility is executed rather than asserted: a cold run attempts the probe on a fresh seed with nothing planted, and every check it fails becomes trap-eligible. No check is a trap because the author says so. The recall side works the other way around, and that asymmetry is where author judgment enters: plant-only facts are declared in each chain's manifest and then certified by a cold run that can demote a declared fact but never promote one. Five facts went unrecovered by a cold worker and still count as secondary, because nobody declared them.
Then the whole instrument moves when the worker changes. Swap in a related model from the same family and one chain stops being a trap, because a cold run of that model declines the memoization unaided; two declared facts get demoted because it derives them from the checkout without the plant. Same suite, same prompts, same graders. Eligibility is model-relative, and every number built on it inherits that.
The protocol was frozen before the confirmatory run, with hypotheses, ranked metrics, a rerun rule, and a manifest pinning every suite-defining file by SHA-256. Execution after that point is genuinely constrained and anyone can check it. It is also worth less than it sounds. The five chains are development-exposed: the product changed in response to failures on these same chains, and the confirmatory run reuses them, so the freeze confirms disciplined execution rather than unseen tasks. The same person wrote the package, the chains, the traps, the graders and the plant-only declarations. The freeze carries no third-party timestamp, so it is pre-specified rather than pre-registered. And eligibility rests on one or a few stochastic cold executions: one chain's cold control ran three times and disagreed with itself, passing, then failing two checks, then passing again, under a harness with no rule for adjudicating repeats. Scoring the alternative leaves the ordering unchanged. Eligibility nonetheless rests on a small and unstable set of cold runs.
One cell, end to end
One cell of the benchmark, from a fresh seed worktree through four sessions that share it: plant, distract, probe and recall. The hidden grader runs against a post-probe snapshot and the chain's pristine external copy. The pinned judge grades each recall fact in its own call. Two cold controls run on fresh seeds with nothing planted: a probe-cold run whose failures define the trap set, and a recall-cold run that certifies or demotes the declared plant-only facts.
The instruments live outside the agent's reach in different ways, and the difference matters. The external enforcement package is copied into the cell beside the worktree and the agent can read and run it, because the prompts reference it by relative path; grading uses the chain's pristine copy instead, so a session told the external is broken cannot help itself by editing it. The hidden grader is never placed in the cell at all. The two dashed boxes at the bottom are the contested part: neither is a thought experiment, both are executed runs, and between them they decide which checks and which facts are allowed to count at all. Hover any box for what it does. Illustrative visualization.
Change the worker, and the instrument changes with it
Per chain, trap eligibility and the surviving plant-only facts, under two worker models side by side. Under Luna, four of five chains carry a trap endpoint and all nine declared plant-only facts survive certification. Under Sol, memo-poison loses its trap entirely, and two declared facts are demoted because a cold Sol recovers them from the repository alone. The tasks are identical in both columns.
Read across each row. Nothing about the suite changes between the two columns: same chains, same prompts, same graders, same hidden checks. What changes is what those instruments can measure, because both eligibility sets are properties of the model that was run against them, and a chain whose trap a cold worker walks past unaided was never testing memory. Hover any row for the check names behind each mark and the facts a cold Sol recovered. Note also the state that cannot occur: a fact nobody declared can never be promoted into the headline metric, however clearly a cold run failed to recover it. That is the asymmetry between a derived trap set and an authored recall set, and it is the most direct route by which the benchmark author's judgment reaches a headline number. Measured from the sealed run artifacts.
What Is Next
The open problem is write-side quality, not recall coverage
The write-desk finding above sets the agenda. Getting knowledge back is the part that works; getting it written down faithfully, and keeping it faithful across rewrites by agents who each believe they are improving the page, is the part that does not. The laundering guard is a first answer and an incomplete one, since it can quote a dropped line back but cannot make an agent keep it, and the quote it sends is a 160 character prefix that often stops before the clause that mattered.
Three experiments follow from that, and each targets a limit named on this page rather than a feature. Unseen chains, because everything here was measured on tasks the product was developed against. A second judge, because the recall ordering currently moves on one judged item. And an ablation separating the store from the surfacing layer, because this evaluation is package-level by construction and cannot yet say which half is carrying the result.
Embedding-based recall is the obvious mechanism to add, and it is deliberately absent. Deterministic resolution answers the asset-scoped case without a similarity threshold to tune, and the case it does not answer, knowledge that belongs to no asset, is the one worth having evidence about before building for it.
That programme has since run. Part 2 prices touch and recommendation directly and bounds search through the arm that carried it, across six preregistered studies and 679 graded cells, and reports what shipped, what ships off, and what got deleted by its own measurements.
pi-canon is published on npm as pi-canon under the MIT license. The benchmark, its drivers, the frozen protocol, and the verifier that recomputes the paper's 92 quantitative claims from the artifacts are in a separate repository. Both are linked below.
Citations
The paper, the package, the benchmark, and the work this stands on
The paper. Shane Conner, "pi-canon: Mutable Canonical Memory over an Immutable Journal, with Recall by Surfacing. Design, and a four-arm multi-session evaluation", Zenodo, August 2026. doi.org/10.5281/zenodo.21890648. The deposit beside the PDF carries the per-cell artifact trail for every cited run, the figure sources, and a provenance index mapping every claim in the paper to the artifact field behind it.
pi-canon. The package: github.com/shaneconner/pi-canon. Published on npm as pi-canon, MIT licensed.
canon-bench. The five chains, the drivers, the frozen protocol with its hash manifest, and the results: github.com/shaneconner/canon-bench. tests/verify_claims.py recomputes all 92 quantitative claims in the paper from the shipped artifacts and names the source field for each.
The narrative version. Shane Conner, "My agents' wiki was written faster than it was read, and what was read sold me back debt I had already paid", Medium, August 2026: medium.com/@shane.conner/my-agents-wiki-was-written-faster-than-it-was-read. How the campaign got here, in prose.
pi-fold. The companion package on the working tier, separate and optional in both directions: shaneconner.com/projects/pi-fold.
Episodic and semantic memory. Endel Tulving, "Episodic and Semantic Memory", in Organization of Memory, Academic Press, 1972. The two halves of long-term declarative memory that the journal and the canon map onto.
Working memory. Alan Baddeley and Graham Hitch, "Working Memory", in The Psychology of Learning and Motivation, 1974; and Richard Atkinson and Richard Shiffrin, "Human Memory: A Proposed System and its Control Processes", 1968.
Letta, formerly MemGPT. github.com/letta-ai/letta. The canonical agent-held memory verbs, where storage addresses are the model's choice and recall is a search the model must think to run.
The AGENTS.md convention. agents.md. The always-in-context convention file, which is the arm this evaluation runs against.
Lost in the Middle. Nelson Liu and colleagues, 2023. arXiv:2307.03172. Retrieval quality degrading with context occupancy, which is the failure mode an always-loaded convention file walks into at scale.