# the infinite window

> a bounded context window, filled from memory on every turn, measured against sending the whole history

innernet labs · white paper 02 · measured · updated 2026-10-26

https://innernet.live/labs/papers/infinite-window

---


## abstract

a model sees only its context window, so every memory system decides which tokens go there. the common answers send
the whole history on every call, 172,871 tokens a question on average on LongMemEval-S, or retrieve a few passages per
ask. we describe a bounded window: on every turn, a packet of the memories the ask needs, sized by the ask and not by
the memory, with handles for what the window already holds, a cache-stable prefix and compaction as a backstop. we
compare it with the whole history under one reader, Claude Haiku 5.5, judged by Claude Sonnet 5.5. on 100 held-out
LongMemEval-S questions, the window at k=40 answered 90.0% right, as the whole history did (paired +2 / -2, exact
McNemar p 1.000), on 25.9 times fewer task tokens. on 90 LoCoMo asks, replayed as multi-turn chats over three
conversations of about 20 to 40 thousand tokens, it lost: 73.3% against 87.8% for the whole conversation kept in cache
(p 0.004). between those two sizes we put a switch at about 100k tokens, which neither run locates. the same work found
and fixed two faults in innernet's production read path. on LongMemEval-S only the packet is exercised; the window's
other parts ran only on LoCoMo, where it lost. the LoCoMo test treats asks from three conversations as independent,
and the judge, from the reader's model family, was not validated here. a window whose cost stays flat as memory grows,
the infinite of the title, is not yet measured.

## 1. introduction

a model reads what is in its context and nothing else, so every memory system answers one question, whether it says so
or not: which tokens to put there. two answers are common. the first sends everything: the whole history, every turn,
re-sent on every call, until the history outgrows the window. the second retrieves: a search over the memory picks a
few passages per ask and sends those. both treat the window as the place where memory lives.

we treat it the way an operating system treats ram (Denning 1968). the store is elsewhere, large, durable and owned by
the person. the window is small and temporary, and its contents are chosen again on every turn by a step whose only job
is to choose. we call this the infinite window because the window does not grow: what grows is the memory behind it,
and the window holds what the ask in front of it needs. the name states the design goal. what this paper measures is a
bounded window against the whole history, at two sizes of memory.

main contributions:

- **a bounded window.** a design in which the packet is sized by the ask and by k, never by the memory, with handles, a send-once ledger, a cache-stable prefix, an intent block and compaction as a backstop; each component is marked measured, built or designed (section 3, table 3.1).
- **the whole history's accuracy on 25.9 times fewer task tokens.** on 100 held-out LongMemEval-S questions the window at k=40 answered 90.0% right, as the whole history did, and an ask cost $0.00083 against $0.0870 sent fresh (5.1, 5.2).
- **a loss on LoCoMo, and where we put the switch.** on LoCoMo a whole conversation kept in cache beat the window, 87.8% against 73.3% (p 0.004); between the two runs' sizes we put a switch at about 100k tokens, which neither run locates (5.3, 5.4).
- **two faults in a production read path.** the read could choose a gold page for 2 of 22 labelled asks and the selector kept none of the sections it named on 149 production calls; after the fixes, 22 of 22 and 3 of 3 (5.5).

section 2 states the problem, 3 the design, 4 the runs, 5 the results, 6 related work, 7 the limitations and 8 the
conclusion. appendix a holds speed measurements outside the claim, and appendix b the editions.

## 2. problem

### 2.1 what is decided

on every turn, a memory system chooses which of the person's stored sections enter the context window. we judge the
choice by three things: whether the answer is right, how many tokens the turn sends, and what the turn costs. sending
the whole history hands the reader everything and lets the second and third grow with the history. retrieval bounds
the tokens and risks the answer. the question this paper asks is whether a window bounded by the ask keeps the
accuracy of the whole history, and at what size of memory the whole history is the better choice.

### 2.2 the paging analogy

the design borrows virtual memory's terms, and they are used in this sense below: the window is the cache, the
person's memory is the disk, innernet is the pager, and a fast decider, the system one (3.8), chooses on every turn
what moves from the disk into the window and what stays out.

### 2.3 what was known before these runs

- **the read path was blind to most of a large memory.** the step that chose which sections answer an ask saw only the first 140 sections of a memory, in load order. whatever came after the 140th could not be chosen, however well it answered. section 5.5 measures what that cost.
- **a small packet can beat the whole history, in one setting.** in [white paper 01](/labs/papers/person-models), on 40 synthetic people, pre-registered and scored once, netti-0's packet did 0.140 nats better than the person's whole history in context at 84.6 times fewer served tokens on the questions scored (100.2 times over every read), and Claude Haiku with the packet beat Claude Opus with no memory by 0.30 nats on 10 people (its H9). a token there is a character count divided by three, not the API's count used here (4.1). two of its guardrails, G3 and G4, failed. that is a person model on a benchmark of people who are programs; it is why bounded packets were worth measuring here, not evidence that they work here.

## 3. method

the window is a set of components around one packet. table 3.1 says where each one stands; each subsection ends in
the same status.

| component | where it is built | what is measured |
|---|---|---|
| a packet bounded by the ask | both runs | accuracy, tokens and cost against the whole history |
| handles, as labels | the LoCoMo replay | a count of compacted memories re-sent in full when retrieval brought them back; no ablation |
| a reader asking for a handle | nowhere yet | nothing |
| a send-once ledger | the LoCoMo replay | part of the window arm; no ablation |
| diffs | nowhere yet | nothing |
| a cache-stable prefix | the LoCoMo replay | the share of prompt tokens read from cache |
| compaction | the LoCoMo replay, clearing the chat so far | a count of compactions; no ablation, so its share of the lost asks is unknown |
| the intent block | the LoCoMo replay, in a short form, in each turn's message and not in the prefix | nothing on its own |
| pre-built packets | nowhere yet | nothing |
| the decider | alone, outside the door | its speed, success and price alone |
| whole while small | the branch window-door, not live | nothing as a switch |

*table 3.1 · the components of section 3 and their status. read across: a component built in a run but not ablated is part of what that run's window arm measured, never measured on its own. what the window arm measures as a whole is in 5.1 to 5.4 · source: packages/bench/infinite/lme-window.mjs and session-replay.mjs (PR #275), and section 4*

### 3.1 a packet bounded by the ask

on each turn innernet sends a packet: the sections of the person's memory this ask needs, best first. its size is set
by the ask and by k, the number of memories it brings in, never by the size of the memory. by design, a small memory
and a very large one produce packets of the same order for the same question. the packets came to 1,693 to 13,982
prompt tokens a question on LongMemEval-S as k ran from 10 to 100, and 7,437 a turn on LoCoMo (5.1, 5.3).

status: measured (5.1, 5.3).

### 3.2 handles

a section the window already holds is not sent again. it is named by a handle: a short, stable reference the model can
use and the pager can resolve. a handle costs a few tokens where the section cost hundreds. by design, when the model
needs the words again it asks for the handle, and only then are they sent.

status: handles as labels are built in the LoCoMo replay, where a compacted memory is re-sent in full, under its
handle, when retrieval brings it back; counted, not ablated (5.3). a reader asking for a handle is designed, not built:
the replay's reader has no way to ask for one.

### 3.3 a send-once ledger and diffs

innernet keeps, per session, a ledger of what the window holds. a section is sent once. if it has not changed, every
later turn refers to it by handle. if it has changed, the change arrives as a diff against what the window holds: the
line that moved, not the page it sits on. by design, the session's cost then grows with what is new, not with how long
it has run.

status: the ledger is built in the LoCoMo replay, as part of the window arm (5.3). diffs are designed, not built in any
run here.

### 3.4 a cache-stable prefix

providers cache a prompt's prefix: if the first part of a request is byte-identical to a recent one, they read it from
cache at a fraction of the price and latency (Gim et al. 2024; Anthropic 2024). so the first part of what innernet sends
is kept stable: who the person is, how they work, the standing instructions. it changes only when those change, and
then once, so the cache is warm for every turn that follows. what varies, the turn packet, comes after it.

status: built in the LoCoMo replay, where new memories are appended in arrival order so the prefix stays byte-stable;
measured as the share of prompt tokens read from cache (5.3).

### 3.5 compaction as a backstop

long sessions still get long. when a window nears its limit, the oldest turns can be compacted into a summary. here
that is a backstop: by design, because what matters is kept in the memory and brought back by the pager, a compaction
that drops a detail loses nothing that cannot be fetched again on the turn that needs it.

status: built in the LoCoMo replay, where the window compacts when the memories it holds would pass 8,000 tokens,
counted as characters ÷ 3. a compaction there clears the chat so far and keeps every held memory's handle, a short stub
for each of the 40 retrieved most often, the open threads and the last exchange; earlier answers are not kept.
counted, not ablated (5.3): whether compaction cost any of the asks the window lost is not measured.

### 3.6 the intent block

a short block at the top of every packet says what the person is doing now: the thread they are in, what they asked
last, what is open. it is meant to keep a new tool, or a model that has just compacted, pointed at the same work. by
design it is rebuilt when the intent changes and otherwise left alone, so it stays inside the cached prefix.

status: built in the LoCoMo replay in a short form (the ask, the held memories that bear on it by handle, and open
threads from earlier turns). there it is rebuilt every turn in the current message, after the cached prefix, not held
in it. not measured on its own.

### 3.7 pre-built packets

between sessions, while nothing is being asked, netti builds the packets the next session is likely to need: the open
threads, what changed overnight, yesterday's corrections. the first turn of the morning then starts from a packet that
is already made, rather than from a search.

status: designed; not built in any run here.

### 3.8 the decider

the components above turn on one decision, made many times a turn: does this section move into the window, stay out, or
go in as a handle? a reasoning model could make it, but slowly and at a price that grows with the memory. the design
makes it with a system one, after Kahneman's (2011) fast system: a decider that reads a state and answers typed
questions (which of these, yes or no, how much) with calibrated probabilities. by design, code stays in control: the
decider's answer is one input, every call has a deadline, and when it misses the deadline or fails, the door falls back
to what it does today, so a slow or absent decider makes the packet worse, never the turn late.

status: measured alone, outside the door (5.6). none of the runs on the door uses it (4.3).

### 3.9 whole while small, the window past it

the runs in section 5 led to one rule the design did not have: while a memory is small, send it whole and let the
provider's cache keep it; past about 100k tokens, the bounded window. the line at 100k is ours, put between the two
sizes the runs measured (5.4).

status: the door is built to make that switch on its own, on the branch window-door, and it is not yet live. no run
here measures the switch.

## 4. setup

### 4.1 LongMemEval-S: the whole history against the window

LongMemEval-S asks questions over long chat histories (Wu et al. 2025). we took 100 held-out test questions, balanced by
type (17 single-session-user, 17 multi-session, 17 single-session-preference, 17 temporal-reasoning, 16
knowledge-update, 16 single-session-assistant), and ran five arms on the same reader, Claude Haiku 5.5, with the same
answer rules and item format in every arm: the whole history in context, and the window at k=10, 20, 40 and 100, where
k is the number of memories the window brings in for the ask. the four k values were fixed before the run. each
question is a single ask, so of section 3 only the bounded packet (3.1) is exercised: the window arm is the search of
innernet's 91.6% LongMemEval-S run at a fixed k, with its query expansion (packages/bench/BENCHMARKS.md, section 00).
the judge is LongMemEval's own prompt on Claude Sonnet 5.5, a model from the reader's family.

tokens are the API's own count (countTokens). task tokens add the expansion call the window runs on every ask, input
and output. accuracy carries a 95% Wilson interval over the 100 questions (Wilson 1927), and each arm is paired with
the whole history on the same questions: the ones it got right that the whole history missed (+), the reverse (-), and
an exact McNemar p (McNemar 1947). we time the answer call alone and the whole ask path, which adds the expansion and
the search before it, and we price each ask. ingest, cutting a history into memories, is paid once per history and
priced on its own. as a cross-check, the k=100 arm's verdicts agree with the shipped 91.6% run on 94 of the same 100
questions. the run cost $9.00.

### 4.2 LoCoMo: sessions replayed

LoCoMo holds long conversations between two people (Maharana et al. 2024). we replayed three of them, conv-26 (about
26,964 tokens), conv-30 (about 20,079) and conv-41 (about 40,139), as true multi-turn chats, 30 asks in a row each, 90
turns in all, three ways: rag (a fresh top-k and the chat so far), the whole conversation kept in the provider's cache,
and the infinite window at k=20, which compacts when the memories it holds would pass 8,000 tokens and re-sends a
compacted memory in full, under its handle, when a later ask retrieves it (packages/bench/infinite/session-replay.mjs).
that cap is counted as characters ÷ 3; every token count in 5.3 is the API's own. on every turn we record the prompt,
the part of it that was new and the part read from cache. the reader is Claude Haiku 5.5; the judge is a plain correct
or wrong prompt written for this run, not LoCoMo's own scoring, on Claude Sonnet 5.5. accuracy carries a 95% Wilson
interval over the 90 asks, and the pairing with the whole conversation is an exact McNemar test on the same asks. both
treat the 90 asks as independent, though the 30 asks of one conversation are turns of one chat that carry state from
turn to turn. the run cost $0.20.

### 4.3 the production door

innernet's own map, 1,551 sections, with 22 asks labelled with the pages that answer them, their gold pages. the
measure is contextbench recall@N: does the set of sections the read can choose from hold a gold page for the ask? no
jev in this run. beside it, the llm selector's own record in production, a live A/B of its fix, and live asks on
innernet.live once both fixes were in. production reads are counted, never read: the measures are sizes and times, not
anyone's words.

### 4.4 the decider

a load run against jev 1.14 alone, through OpenRouter, outside the door, on 15 and 16 october 2026: 100 calls, timed
and priced. beside it, the Artificial Analysis Decision Index v1.0 of 13 october 2026, reported by them and not measured
here (Artificial Analysis 2026). it rates deciders on its own tasks; table 4.1 gives the three it reports that this
paper names.

| decider | Decision Index v1.0, reported | time a task, reported |
|---|---:|---:|
| Claude Haiku 5.5, low | 61 | 1.4 s |
| jev 1.13 | 58 | 0.13 s |
| Microsoft decision-1 | 54 | · |

*table 4.1 · the Artificial Analysis Decision Index v1.0, 13 october 2026, as they report it; none of it is measured here. read the score higher as better and the time lower as better; a dot means no time is reported. jev 1.13 is three points under Claude Haiku 5.5 at about a tenth of its time a task · source: Artificial Analysis 2026, and content/labs/data/infinite-window.json*

## 5. results

every cell in this section comes from a run, named under its table. none is filled by hand.

### 5.1 accuracy against the whole history

at k=40 the window answered 90.0% of the 100 questions right, the whole history's accuracy with the same interval, on
25.9 times fewer task tokens. the pairing finds four questions where the two disagree, two each way (exact McNemar p
1.000).

| arm | prompt tokens / q | task tokens / q | fewer task tokens | accuracy [95% CI] | paired with the whole history | answer, p50 / p95 | ask path, p50 / p95 | $ / ask |
|---|---:|---:|---:|---|---|---:|---:|---:|
| the whole history | 172,871 | 173,198 | · | 90.0% [82.6, 94.5] | · | 4.75 s / 6.99 s | 4.75 s / 6.99 s | $0.0870 |
| the window, k=10 | 1,693 | 2,385 | 72.6× | 81.0% [72.2, 87.5] | +3 / -12, p 0.035 | 2.21 s / 5.15 s | 3.90 s / 6.70 s | $0.00039 |
| the window, k=20 | 3,142 | 3,871 | 44.7× | 86.0% [77.9, 91.5] | +1 / -5, p 0.219 | 2.38 s / 5.84 s | 4.10 s / 7.50 s | $0.00055 |
| the window, k=40 | 5,939 | 6,675 | 25.9× | 90.0% [82.6, 94.5] | +2 / -2, p 1.000 | 2.46 s / 5.67 s | 4.22 s / 7.32 s | $0.00083 |
| the window, k=100 | 13,982 | 14,766 | 11.7× | 89.0% [81.4, 93.7] | +2 / -3, p 1.000 | 2.81 s / 6.22 s | 4.49 s / 7.91 s | $0.00166 |

*table 5.1 · LongMemEval-S, 100 held-out test questions, the whole history and the window at four k values fixed before the run. read each window row against the first: accuracy with its 95% Wilson interval, the paired count (+ right where the whole history was wrong, - the reverse) with its exact McNemar p, and "fewer task tokens" as the whole history's task tokens over the arm's; a dot means the column does not apply. k=40 is the smallest of the four k values that kept the whole history's accuracy, at 25.9 times fewer task tokens · reader Claude Haiku 5.5, judge LongMemEval's own prompt on Claude Sonnet 5.5 · source: packages/bench/results-infinite-window-lmes-haiku-5-5.json and results-infinite-window-report.md (PR #275), BENCHMARKS.md section 00*

k=100 is one question short of the whole history (89.0%, p 1.000). fewer memories cost answers: k=20 lost 5 and gained
1, which the pairing cannot tell from noise (p 0.219), and k=10 lost 12 and gained 3, which it can (p 0.035). the
interval at k=40 is as wide as the whole history's, so the run cannot tell the two apart; that is not proof that they
are the same (7).

### 5.2 the cost of an ask

sent fresh, an ask through the window at k=40 cost $0.00083 against $0.0870 for the whole history, about a hundred times
less. against a whole history kept in a provider cache, an estimate, it cost about 11 times less. both are before
ingest, which the window pays once for each history it reads.

| what an ask costs | the whole history | the window, k=40 |
|---|---:|---:|
| sent fresh on every ask | $0.0870 | $0.00083 |
| the whole history kept in a provider cache, an estimate | $0.00946, after a first write of $0.1089 | · |
| ingest, paid once per history | none | $0.0287 a question |

*table 5.2 · what one LongMemEval-S ask costs, the whole history against the window at k=40. read down a column: the bill sent fresh, the bill if the history stays in a provider cache, and the one-time ingest; a dot means the row does not apply. the cached row is an estimate for asks that arrive while the provider still holds the history, which narrows the gap from about a hundred times to about 11; ingest is the extraction cost per question from the full-500 run's ledger, and embedding the history is not priced · source: packages/bench/results-infinite-window-report.md (PR #275)*

### 5.3 accuracy on LoCoMo

on LoCoMo the window lost: 73.3% against 87.8% for the whole conversation kept in cache, with 16 asks lost and 3 gained
(exact McNemar p 0.004), on 7,437 prompt tokens a turn against 33,819.

| arm | prompt tokens / turn | new tokens / turn | read from cache / turn | cache share | accuracy [95% CI] | paired with the whole conversation | latency, p50 / p95 | $ / ask | $ / ask with ingest |
|---|---:|---:|---:|---:|---|---|---:|---:|---:|
| rag, a fresh top-k and the chat so far | 2,433 | 1,429 | 1,003 | 41% | 72.2% [62.2, 80.4] | +4 / -18, p 0.004 | 1.11 s / 1.98 s | $0.00018 | $0.00066 |
| the whole conversation, cached | 33,819 | 1,175 | 32,644 | 97% | 87.8% [79.4, 93.0] | · | 1.16 s / 2.02 s | $0.00051 | $0.00051 |
| the infinite window | 7,437 | 1,510 | 5,927 | 80% | 73.3% [63.4, 81.4] | +3 / -16, p 0.004 | 1.10 s / 2.29 s | $0.00029 | $0.00077 |

*table 5.3 · LoCoMo session replay, 3 conversations, 30 asks in a row each, 90 in all, k=20, the window compacting when its held memories would pass 8,000 tokens counted as characters ÷ 3. read each row against the cached whole conversation: tokens a turn split into what was new and what was read from cache, accuracy with its 95% Wilson interval over the 90 asks, and the paired count with its exact McNemar p; a dot means the column does not apply. both retrieval arms lost to the whole conversation (p 0.004 each), and the window cost more an ask than the whole conversation once ingest is counted · reader Claude Haiku 5.5, judge a plain correct-or-wrong prompt written for this run, on Claude Sonnet 5.5 · source: packages/bench/results-infinite-window-locomo-session.json (PR #275)*

the p of 0.004 treats the 90 asks as independent (4.2). counted per conversation, the window lost in all three: it was
right where the whole conversation was wrong on 0, 2 and 1 asks, and wrong where it was right on 4, 6 and 6 (conv-26,
conv-30 and conv-41; source: packages/bench/results-infinite-window-locomo-session.json).

80% of the window's prompt tokens were read from cache, though its new tokens a turn, 1,510, were more than the cached
whole conversation's 1,175. it compacted 9 times and re-sent 393 compacted memories in full when retrieval brought them
back. at this size the
window was no better than rag, and the whole conversation was not expensive: a cached prompt bills at a fraction of the
fresh price, so it cost $0.00051 an ask against the window's $0.00077 once ingest is counted.

### 5.4 the size rule

**while memory is small, send it whole and keep it cached; past about 100k tokens, the bounded window.** the two runs
sit on either side of that line. LongMemEval-S histories are 172,871 tokens a question on average, and there the
window kept the accuracy at a small share of the tokens. LoCoMo conversations are about 20,079 to 40,139 tokens, and
there the whole conversation won. the runs do not place the line more finely than that: 100k is where we put the
switch between two measured sizes, not a threshold either run found. the two runs also differ in more than size: the
benchmark, one ask against 30 asks in one chat, a whole history sent fresh against one kept in cache, and the judge's
prompt. so size is not the only thing that changed between them (7).

### 5.5 two faults in the production read

before the fixes, the read could choose a gold page for 2 of 22 labelled asks, and the llm selector had kept no section
it named on any of 149 production calls in 3 days. after them, the read could choose one for all 22, and the selector
kept 3 of 3 in a live A/B. an ask that found nothing in the morning found the right section later the same day.

| measure | before | after |
|---|---|---|
| asks, of 22, whose gold page the read could choose from | 2, in the first 140 sections in load order | 22, in the BM25 shortlist's top 60 |
| sections the llm selector named and kept | none, on any of 149 production calls in 3 days | 3 of 3 in a live A/B, where the old format kept 0 of 2 |
| the selector's answer, output tokens | 114 | 22 |
| an ask that found nothing in the morning | 0 focus sections | the right section: 2 focus sections, about 363 tokens, chosen from 1,591 sections, the count the live read considered that day |
| a later live ask | · | the llm selector ran: 3 focus sections, about 1,566 focus tokens, the select step 2.1 s |

*table 5.5 · innernet's own map, 1,551 sections, 22 labelled asks, contextbench recall@N, no jev. read each row before and after the two fixes: the first row is the bench, the selector rows are its production record and a live A/B, and the live rows are innernet_context's own sent line on innernet.live, 16 october 2026, where the read considered 1,591 sections against the bench's 1,551; a dot means there was nothing to measure. both faults were real and both are fixed, on one map · source: PRs #272 and #274*

the first fault was a window into the map that never moved: whatever sat past the 140th section could not be chosen.
the shortlist now ranks every section, BM25 over the words (Robertson and Zaragoza 2009) fused with cached vectors, and
hands the selector the best first. the second was quieter. section names hold dashes and dates, and asked to copy a name
back, the model copied the whole line, so an exact-match guard dropped every name it gave. it now answers with short
ids, mapped back in code, in a fifth of the output tokens.

### 5.6 the decider's speed

alone and outside the door, jev 1.14 through OpenRouter answered at a p50 of 0.49 s, and 100 of 100 calls succeeded at
about 1,160 a minute with no throttling. it costs $0.042 per million input tokens, with output free, and every request
asks for zero retention. the routes before it were not always there: on the Vercel gateway the free-tier key got 403,
and in september the shared credentials walled at about 12 context asks a minute.

Artificial Analysis reports jev 1.13 three points under Claude Haiku 5.5 at about a tenth of its time a task (table
4.1). we read a decider that fast as one that can sit on every turn; no run here puts it there. we also ran a
classifier of our own, Laya (ModernBERT-large, self-hosted; Warner et al. 2025). it got 14 of 19 context decisions
right, and it is retired.

## 6. related work

### 6.1 virtual context and paging

MemGPT (Packer et al. 2023) treats the context window as main memory and lets the model move information between it and
external storage through function calls, after operating-system virtual memory. paging also manages the attention cache
in serving (Kwon et al. 2023), and keeping only a working set in fast memory goes back to Denning (1968). here code, not
the reader, decides what enters the window, and the window is bounded by the ask.

### 6.2 retrieval

retrieval-augmented generation fetches passages for each query and conditions the answer on them (Lewis et al. 2020),
and memory layers such as Mem0 extract and retrieve memories from a conversation (Chhikara et al. 2025). on
LongMemEval-S our window arm is retrieval at a fixed k; what this work adds is the multi-turn state around it (handles,
the ledger, the cached prefix) and a comparison against the whole history under the same reader on the same items.

### 6.3 prompt compression and compaction

prompt compression shortens what is sent by dropping tokens a small model judges unimportant (Jiang et al. 2023), and
agents compact long sessions into summaries. here compaction is only a backstop: a compacted memory keeps its handle and
comes back in full when a later ask retrieves it.

### 6.4 prompt caching

prompt caching reuses the computation for repeated prompt segments (Gim et al. 2024), and providers bill a cache read at
a fraction of the fresh input price (Anthropic 2024). the window keeps its prefix byte-stable so that a provider cache
serves it. the LoCoMo run shows the other side: caching also makes the whole-conversation baseline cheap (5.3).

### 6.5 long-context evaluation

models use information in the middle of a long context worse than at its ends (Liu et al. 2024), and BEAM builds
conversations of up to 10M tokens to test memory beyond the window (Tavakoli et al. 2026). we did not test position
effects. BEAM is where a flat cost as memory grows would be measured (8).

### 6.6 memory benchmarks

LongMemEval (Wu et al. 2025) asks questions over long chat histories across several memory abilities, and LoCoMo
(Maharana et al. 2024) asks them over long conversations between two people. we use LongMemEval-S as single asks and
replay LoCoMo's questions as one multi-turn chat per conversation, and pair every arm with the whole history on the
same items.

## 7. limitations

- **100 questions is a small sample.** every 95% interval is 12 to 15 points wide, so at k=40 the run could not tell the two arms apart, and that is not proof that they are the same. it bounds 5.1 and the second contribution. closing it: the same arms on more of LongMemEval's questions.
- **one reader, and a judge from its family, not validated.** every arm was read by Claude Haiku 5.5 and judged by Claude Sonnet 5.5. a stronger or weaker reader may use a whole history and a packet differently, and a judge from the reader's family is not independent of it. neither judge was checked here against people or against a judge from another family: LongMemEval-S used its own published prompt, and LoCoMo a plain correct-or-wrong prompt written for this run. it bounds every accuracy here; the headlines are pairings of two arms under one judge, not accuracy levels. closing it: a reader and a judge from another model family on the same items, with their agreement reported as Cohen's κ.
- **k was chosen before the run, not tuned.** k=40 is the best of four values fixed in advance, not an optimum.
- **LongMemEval-S tests only the packet.** each question is one ask, so handles, the ledger, the cached prefix and compaction (3.2 to 3.5) are measured only on LoCoMo, where the window lost. it bounds the first contribution: no run shows those components helping.
- **LoCoMo is three conversations.** its 90 asks are 30 turns in each of 3 chats, and both the Wilson intervals and the exact McNemar p treat them as independent, though turns of one chat carry state between them. the window lost in each of the three conversations (5.3), but three are too few to resample by conversation. it bounds 5.3, the third contribution and, with one benchmark on each side of it, the size rule in 5.4. closing it: more conversations, and an interval resampled by conversation.
- **cheaper, not much faster.** the window's answer call is about twice as fast (p50 2.46 s against 4.75 s), but the expansion and the search give most of that back: the whole ask path is only about 11% faster (4.22 s against 4.75 s).
- **the dollar gap depends on caching.** sent fresh, the window is about a hundred times cheaper an ask. against a whole history kept in a provider cache it is about 11 times, and the cached figure is an estimate. both are before ingest, $0.0287 a question, which the window pays once per history, and embedding the history is not priced.
- **on LoCoMo the window lost.** 73.3% against 87.8%, and no better than rag. in this run, memory of 20 to 40 thousand tokens was better sent whole and cached.
- **the switch is placed, not tested.** the door's switch between whole and window is built on a branch and not live, and no run here measures it (3.9). the two runs differ in benchmark, in task (one ask against 30 in one chat), in how the whole history was sent (fresh against cached) and in judge prompt as well as in size, so they do not show that size is what should decide (5.4). closing it: the switch measured live, at sizes between the two runs, on one benchmark.
- **the door's numbers are one map.** innernet's own, with 22 labelled asks and a few live asks: they show that both faults were real and are fixed, not how the door does across many people's memories.
- **the desk is a fake desk.** its hand time is modelled, so the numbers in appendix a.2 say how the decisions behave, not how fast a real Mac's hands are.
- **jev did not make ask netti's first word sooner.** 2.63 s with it, 2.29 s without (appendix a.1).
- **nothing about how it feels.** whether a person notices a window that does not fill, and whether they trust it, is not measured here.

## 8. conclusion

**bounded window, unbounded memory, flat cost.** that is the claim this paper is building toward, and the runs say
where it holds today. on histories of 172,871 tokens a question on average, the window kept the whole history's
accuracy at k=40 on 25.9 times fewer task tokens, at about a hundredth of the price while the history is sent fresh,
before ingest. on conversations of 20 to 40 thousand tokens, it lost to sending the whole conversation and keeping it
cached. so the door is built to do both, on the branch window-door and not yet live: while memory is small it is sent
whole and cached, and past about 100k tokens the window takes over.

infinite here means a window whose cost does not move as memory grows: success, tokens per turn and the wait for the
first token all stay flat as a history grows from 10k to 10M tokens. until that is measured, infinite is what we are
building toward, not what we have shown.

future work:

- the flat-cost measurement on BEAM, as a history grows from 10k to 10M tokens.
- the switch between whole and window, live, and measured at sizes between the two runs here.
- an ablation of each component built in the LoCoMo replay (handles, the ledger, the prefix, compaction), and diffs and pre-built packets built and measured.
- the same items read and judged by another model family, with intervals resampled by conversation on LoCoMo.

## appendix a · other speed measurements

these two runs are outside the paper's claim. they are kept here because they share its decider.

### a.1 ask netti

ask netti answers a person's questions about their own memory. running its grounding beside the reading, rather than
before it, brought its first word in sooner.

| measure | before | after |
|---|---:|---:|
| first word, p50 | 4.23 s | 2.29 s |
| first word, p90 | 6.30 s | 4.04 s |
| whole answer, p50 | 4.55 s | 3.70 s |
| grounding, p50 | 487 ms | 84 ms |
| cost, the bench's own line | $0.0548 | $0.0330 |
| cases passing, of 51 | 48 | 50 |

*table a.1 · ask netti, 51 cases, Claude Haiku 5.5, database lag 40 ms, cold. read each row before and after grounding moved beside the reading; lower is better except for cases passing. the first word came sooner at both percentiles shown, and 50 of 51 cases passed, against 48 · source: PR #276*

with jev live in the same path, 48 of 51 passed, and jev did not bring the first word sooner: 2.63 s against 2.29 s.

### a.2 the desk's hands

netti's hands on the Mac desk carry out click tasks. here jev took the steps it could on a fast path, and Haiku the
rest.

| measure | Haiku | Haiku and jev |
|---|---:|---:|
| tasks finished, of 18 | 18 | 18 |
| median passing run | 4,849 ms | 4,805 ms |
| steps a second | 0.62 | 0.64 |
| cost of a passing run | $0.0014 | $0.0009 |
| steps on the fast path | · | 20% |
| jev calls that fell open on a cold 1.2 s timeout | · | 29% |

*table a.2 · a fake desk, six click tasks three times each, hand time modelled. read the two columns against each other; a dot means the row does not apply to Haiku alone. jev made the hands cheaper, not faster: the same 18 of 18 at about the same pace, $0.0009 a run against $0.0014 · source: PR #277, which ships dark*

## appendix b · editions

- **v1, 16 october 2026.** the runs and the text.
- **v1, restructured, 26 october 2026.** the same runs and numbers, in the house structure of content/labs/STYLE.md: an abstract in five moves, a contributions list, a status line for each component that says what the LoCoMo replay built and what it did not, one finding per results subsection, captions that say how to read each table, Artificial Analysis's reported index moved to setup (table 4.1), related work, references and the limitations of LongMemEval-S's single asks, LoCoMo's clustered asks and the unvalidated judges. no number changed. one count is new: the window's LoCoMo pairing per conversation (5.3), from the same run's file.

## references

- Anthropic (2024). Prompt caching with Claude. Anthropic news, 14 august 2024. anthropic.com/news/prompt-caching, now claude.com/blog/prompt-caching.
- Artificial Analysis (2026). Artificial Analysis Decision Index v1.0. 13 october 2026. artificialanalysis.ai/articles/artificial-analysis-decision-index.
- Chhikara, P., Khant, D., Aryan, S., Singh, T., & Yadav, D. (2025). Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory. arXiv:2504.19413.
- Denning, P. J. (1968). The Working Set Model for Program Behavior. Communications of the ACM, 11(5), 323–333.
- Gim, I., Chen, G., Lee, S., Sarda, N., Khandelwal, A., & Zhong, L. (2024). Prompt Cache: Modular Attention Reuse for Low-Latency Inference. MLSys 2024. arXiv:2311.04934.
- innernet labs (2026). person models. white paper 01. [innernet.live/labs/papers/person-models](/labs/papers/person-models).
- Jiang, H., Wu, Q., Lin, C.-Y., Yang, Y., & Qiu, L. (2023). LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Models. EMNLP 2023. arXiv:2310.05736.
- Kahneman, D. (2011). Thinking, Fast and Slow. Farrar, Straus and Giroux.
- Kwon, W., Li, Z., Zhuang, S., Sheng, Y., Zheng, L., Yu, C. H., Gonzalez, J. E., Zhang, H., & Stoica, I. (2023). Efficient Memory Management for Large Language Model Serving with PagedAttention. SOSP 2023. arXiv:2309.06180.
- Lewis, P., et al. (2020). Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. NeurIPS 2020. arXiv:2005.11401.
- Liu, N. F., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Petroni, F., & Liang, P. (2024). Lost in the Middle: How Language Models Use Long Contexts. Transactions of the ACL, 12, 157–173. arXiv:2307.03172.
- Maharana, A., Lee, D.-H., Tulyakov, S., Bansal, M., Barbieri, F., & Fang, Y. (2024). Evaluating Very Long-Term Conversational Memory of LLM Agents. ACL 2024, 13851–13870. arXiv:2402.17753.
- McNemar, Q. (1947). Note on the Sampling Error of the Difference Between Correlated Proportions or Percentages. Psychometrika, 12(2), 153–157.
- Packer, C., Wooders, S., Lin, K., Fang, V., Patil, S. G., Stoica, I., & Gonzalez, J. E. (2023). MemGPT: Towards LLMs as Operating Systems. arXiv:2310.08560.
- Robertson, S., & Zaragoza, H. (2009). The Probabilistic Relevance Framework: BM25 and Beyond. Foundations and Trends in Information Retrieval, 3(4), 333–389.
- Tavakoli, M., Salemi, A., Ye, C., Abdalla, M., Zamani, H., & Mitchell, J. R. (2026). Beyond a Million Tokens: Benchmarking and Enhancing Long-Term Memory in LLMs. ICLR 2026. arXiv:2510.27246.
- Warner, B., et al. (2025). Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference. ACL 2025. arXiv:2412.13663.
- Wilson, E. B. (1927). Probable Inference, the Law of Succession, and Statistical Inference. Journal of the American Statistical Association, 22(158), 209–212.
- Wu, D., Wang, H., Yu, W., Zhang, Y., Chang, K.-W., & Yu, D. (2025). LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory. ICLR 2025. arXiv:2410.10813.