fold Agent-Governed Lossless Context Folding
A session folds its own transcript instead of losing it. The originals stay on disk, verified by hash, one tool call away.
Scroll to explore
The Project
Keep the window small without losing the session
Every long agent session hits the same wall: the transcript outgrows the context window. The standard remedy is compaction, and it is lossier than the word suggests. The session is summarized, the originals leave the model-facing window with no in-band route back, and a new session starts over from the summary, which was written before anyone knew what would matter. fold replaces that with folding: stale spans of the transcript collapse in place into short briefs, while the exact original bytes stay on disk, addressed and verified by SHA-256. Nothing is deleted. What left the window stays one tool call away, and a fold can be peeked or expanded back whole, byte for byte.
The runtime ships on npm as pi-fold under MIT and runs inside Pi. The folding verbs are in-band, so the agent itself can mark what is stale, and an automatic ladder folds on its own when the agent stays silent. A gate suite of over a hundred behavioral contracts is the package's spec, and every measured number on these pages comes from sealed runs whose artifacts ship with the papers.
One workload, two remedies: what the window keeps
Two sessions run the same 64-stage workload. The folding session commits seven times, at stages 12, 23, 35, 45, 55, 60 and 64, applying 13, 15, 12, 15, 11, 10 and 17 folds. Each commit compresses the window in place while the folded originals, 93 by the end, accumulate on disk, hash-addressed and recoverable. It delivers 64 of 64 stages and never pauses. The compacting session pauses three times, at stages 23, 40 and 57, for 44.7, 101.6 and 92.5 seconds, and each pause replaces the window with a summary: 4.0 minutes of stop-the-world in total. The continuation key it needs for stage 57 was stated exactly once, at stage 56. The summary does not contain it, the projection after the boundary holds zero occurrences of it, and the session stops at 56 of 64.
The same workload, run under each remedy. Above, folding: each commit compresses the window in place, and everything folded lands on the recoverable rail, exact bytes on disk behind a hash. Below, native compaction: each pause replaces the window with a summary, and everything the summary dropped is out of the session's reach in-band. The event positions, fold counts, pause lengths, and both endpoints are read from the sealed second session of each arm in the Part 2 campaign, which share one frozen 64-stage workload; the fill drawn between events is schematic. The key that ends the lower session was stated once, at stage 56, and the sealed post-compaction projection provably holds zero occurrences of it. Drag to scrub. Measured events from the sealed run artifacts.
The Evidence
The bill, the memory, the read, the split
The project has run as four measurement campaigns, each published with its paper and full artifact trail. Part 1 built the design and priced it: one controlled pairing against Pi's native compaction on a 64-stage workload, cut at the stage both arms reached. Folding did the same work at 0.599x the tokens and 0.551x the dollars, $11.57 against $21.00, while native compaction restarted the session three times and paid a premium pricing tier on 17 of its requests climbing toward each restart. Part 1 also states plainly what that run did not show: the agent never touched the folding verbs, so the number belongs to the automatic ladder, and the one recall probe the folding arm missed pointed at exactly the question Part 2 was built to ask.
Part 2 asks whether anything is actually lost. It plants seeded values that exist only in the transcript, never in the repository, forces four decisions to be recorded mid-task, and withholds a thirty-cell recall block until all 64 stages are delivered. Both folding sessions finished the withheld block with every recorded decision correct, and the folded bytes were still retrievable at the last request. Neither compacting session finished: both lost a continuation key that had been stated exactly once, and a read-only reconstruction that matches the sealed projection hash shows zero occurrences of the key surviving the compaction boundary. Two sessions per arm is a case series, not a rate, and the paper says so; what it establishes is the mechanism, witnessed end to end under seal. The scored recall block itself was retired by the later campaigns, because a compaction summary can transcribe that kind of exam; Part 3 says why.
Part 1
The Design, and the Bill
The fold, the briefs, the hash-verified way back, and the gate suite that holds it together, then one controlled pairing against native compaction. Folding does the same work for less, the restart tax gets itemized, and the page closes on the gap it leaves open: the agent never used the verbs, so the retrieval half of the design went unmeasured.
Part 2
Working Memory Under Context Shedding
A stress instrument plants values that exist only in the transcript, forces decisions to be recorded mid-task, and withholds a thirty-cell recall block until the work is done. The folding sessions answer all of it. The compacting sessions never reach the block, and the page traces the exact request where the thing they needed disappeared.
Part 3
Ephemeral Retrieval
A read the agent can only look at once: the bytes arrive, the reply becomes the surviving trace, and the page leaves while the fold stays exact. A frontier model takes the short lease on six of its seven reads and chains three large ones at the end of a 65-minute session. Handing a page back turns out to rewrite the whole cached prefix, which is a runtime defect rather than a design one, and the page carries the measurement, the correction to our own first reading, and the repair.
Part 4
Rotation and Retention
Eight sealed sessions work the same long study of a real codebase, then sit the same surprise exam about their own conversation. Compaction answers only what its summary kept, inviting the agent to annotate its own folds pulls wrong answers out of the briefs, and a memory store outside the window lifts both arms to the same ceiling; the fold gets there for half the price. The line closes on its finding: rotation and retention are different jobs.
Papers and Artifacts
Sealed runs, and everything needed to check them
Every campaign is published on Zenodo with its evidence set. The Part 1 paper, design and the trace evaluation, is at doi.org/10.5281/zenodo.21856873, with the figure sources, the redacted per-request ledgers for every cited run, and the full campaign log beside the PDF. The Part 2 paper, the working-memory instrument and the four-session case series, is at doi.org/10.5281/zenodo.21980746, with the frozen plan and seeds, the sealed evidence set, the exact source and Git history, and SHA-256 manifests over all of it. The Part 3 paper, the ephemeral peek and the cache defect that giving a page back exposed, and the Part 4 paper, the rotation and retention split, are at doi.org/10.5281/zenodo.22142454 and doi.org/10.5281/zenodo.22142456. The narrative versions are on Medium: Part 1 as Compaction Doesn't Have to Mean Starting Over, Part 2 as The Summary Kept the Story. It Threw Away the Key, Part 3 as Temporary Context, and Part 4 as A Compaction Summary Is One Record Doing a Store's Job.
The runtime itself is one repository, github.com/shaneconner/fold, which carries the code, the gate suite that is its contract, and the experiment harness that produced every measured number on these pages. It is on npm as pi-fold, MIT licensed. The companion package on the long-term memory tier is canon, separate and optional in both directions.