CASE STUDY 03 — The same workload is worth 19.2% or 4.1%, depending on which cache tier you bought

Quiet Overrun, operated by Local Good Work LLC · 2026-07-29, revised 2026-07-30 · case study v1.2.3 · findings library v1.8.3 · engine cost_model.py, run unmodified. Same pipeline as CASE-STUDY-01: ingest → engine → report, nothing bespoke.

The finding. On one hour of Moonshot AI's Kimi production traffic — the only public production trace that ships prefix-block hashes — measured prefix reuse is 57.1% of tool/agent input tokens and 37.4% of conversation input tokens. What that reuse is worth is not a number. It is a range, and the range is set by a procurement decision most teams make without measuring:

Same traffic. Same reuse. Same measurement. 19.2% or 4.1% — a 4.7× difference decided entirely by which retention tier the buyer is on. The longer tier retains more (hit rate rises from 0.528 to 0.559) but charges 2.00× to write instead of 1.25×, and on this traffic the write premium overwhelms the retention gain.

This is a sensitivity result, not a point estimate. That is the useful form: a reader who does not know which tier they are on now knows the question to ask, and roughly what the answer is worth.

1. The proof underneath: break-even hit rate

Prompt caching is not free. You pay a premium to write a block and a discount to read it, so caching only pays when the hit rate clears

h_break-even = (w − 1) / (w − r)

where w is the cache-write multiple and r the cache-read multiple of base input price. On the cited card (r = 0.1):

• 5-minute tier (w = 1.25): break-even 0.2174. Tool/agent measures 0.5285 — comfortably clear.
• 1-hour tier (w = 2.00): break-even 0.5263. Tool/agent measures 0.5591 — clear by only 3.3 percentage points. Conversation measures 0.3385 — it does not clear break-even at all, and the engine correctly sizes CA-01 at $0 there, leaving only the duplicate-call finding.

That is why the conversational workload's two tiers read 3.49% and 0.66%: on the extended tier, caching this traffic loses money, and the finding's own falsifier says so before a consultant can talk anyone into it.

A workload sitting near break-even is the single most valuable thing this measurement can tell a buyer, and it is invisible without both the hit rate and the tier's write multiple in the same calculation.

2. Both tiers, both expiry semantics, per trace

refresh = a cache read extends the entry's life (the vendor's "hits & refreshes" wording). hard = expiry runs from write time regardless of reads — the pessimistic edge of the band. Everything else is identical.

tool/agent — 23,608 requests, 3537s, mean 8596 in / 182 out tokens

conversation — 12,031 requests, 3537s, mean 12035 in / 343 out tokens

Priced cost before any finding: $19,013.82 per 1M tool/agent requests, $27,496.31 per 1M conversation requests (cited card, this token mix).

3. Why the tier matters so much here: reuse distance

A 5-minute cache cannot serve a prefix last seen 40 minutes ago. How much reuse falls outside the window is a property of the workload, and the two traces differ sharply:

Agentic traffic reuses its prefix immediately — median gap 0 seconds — so a short cache captures nearly all of it. Conversational traffic reuses across minutes, so 21.5% of its reuse falls outside a 5-minute window. This is why agent workloads are the natural caching target and chat workloads are not, and it is measurable from timestamps alone.

4. The reuse measurement — three definitions, and the one that matters

• A — block-occurrence share: reused block occurrences ÷ all block occurrences. A = Σ_r Σ_i [hash seen before] / Σ_r |hash_ids_r|
• B — token-weighted share: input tokens in previously-seen blocks ÷ all input tokens, where block i carries min(512, input_length − 512·i) tokens.
• C — contiguous-prefix share: as B, but a block counts only if every block before it in the same request also hit. This is what a real prefix cache can serve: KV state is positional, so a match at position 5 is worthless if position 4 missed.

C ≡ B exactly, and here is why

Enforcing contiguity changes nothing: C equals B to floating-point precision on both traces. The reason is checkable in one pass of the ingest — zero hits occur after a miss within the same request: 0 of 226,316 hits (tool/agent) and 0 of 105,710 (conversation). The released hash_ids are cumulative prefix hashes, so any reuse is by construction a clean prefix.

That matters because it removes the strongest objection to this method. Block- independent counting could over-credit a cache by counting mid-request matches no positional cache could serve. On this data it provably does not. We publish B because it is what a cache prices on, and C because it is what a cache can serve — and they are the same number.

Against the paper's reported figures (~59% / ~40%): B closes most of the gap that occurrence-counting leaves open. The residual ~2 points is not resolved from the released artifact — the paper measured its production system over data we cannot access; the release is a one-hour sample. We publish what the committed script computes on pinned bytes and do not adjust toward the reported number.

5. Cache pricing is not one number — the card changes the answer

Each row pairs a card with the hit rate measured at its own documented retention window, taken from the ingest's retention probe — pairing a 1-hour hit rate with a 5-minute write price is precisely the error this study was corrected for in v1.1.0. The OpenAI rows are split by model generation because both the write price and the window changed at GPT-5.6. Every multiple and window below was re-read from the vendor's own page on 2026-07-29 (v1.2.0 — the previous OpenAI row was wrong; see changelog).

Three things a reader should take from this table:

1. The reuse ratios and hit-rate curves are card-independent. The savings percentage is not. Conversation swings from 0.66% to 10.56% of spend across published cards on identical traffic. Any vendor quoting a single "caching saves X%" figure without naming a card and a generation is quoting nothing.

1. The cited card is the conservative choice, not a flattering one. Against a no-premium card at the same 0.10× read, Anthropic's 5-minute tier is worse for every hit rate below h = 1.000 — that is, at every hit rate short of a perfect cache. As a share of the input-token bill:

`` Anthropic 5-minute 1 − (0.10h + 1.25(1−h)) = 1.15h − 0.25 no write premium 1 − (0.10h + 1.00(1−h)) = 0.90h equal when 1.15h − 0.25 = 0.90h → 0.25h = 0.25 → h = 1 ``

So pricing this study on OpenAI's pre-5.6 card would have produced a larger savings figure at every hit rate we measured. We kept the smaller one.

v1.1.0 published a different crossover here, because it fed this same equation a misread OpenAI card. The arithmetic was never wrong; the card was. The superseded figures are in the changelog and deliberately not restated here — a live document that repeats its own retired numbers is a document a reader can quote back wrongly. The build now derives the crossover from the card list in closed form and asserts that the closed form still reproduces the old published value when fed the old card, so the correction is provably to the data and not to the mathematics.

1. Identical multiples, different window — and the window saturates. The current GPT-5.6+ card is 0.10× / 1.25×: the same line as Anthropic's 5-minute tier, same break-even, no crossover to compute. The only difference is retention — "at least 30 minutes" against five. That is worth +3.1 points of hit rate on the tool/agent trace (52.8% → 55.9%) and +4.5 points on the conversation trace (29.3% → 33.9%), lifting tool/agent savings from 19.19% to 21.00% of spend.

And no more than that. The measured hit rate is identical at 600 s, 1,800 s and 3,600 s — to the last digit, on both traces — because from ten minutes upward the replay is exactly the untimed, capacity-only replay at the stated 32,000-block reference capacity (0.5591 tool/agent, 0.3385 conversation; asserted in the build, not eyeballed). Past ten minutes this cache is capacity-bound, not time-bound. Buying a longer window buys nothing here; buying more capacity might. That is the opposite of the intuition that a longer TTL is strictly better value, and it is only visible because the window and the price were separated.

On DeepSeek. v1.1.0 carried a "DeepSeek-style" row at 0.10× / 1.00×. With OpenAI's read multiple corrected to 0.10×, that row became numerically identical to the pre-5.6 OpenAI row, so it has been removed rather than double-counted. DeepSeek's own published card is in fact more aggressive than either vendor here — cache-hit input at 0.02× of cache-miss input on V4-Flash and 0.0083× on V4-Pro, so the read multiple is not even constant across their own card — but their pricing page documents no cache-write charge at all. An absence of a documented charge is not a documented zero, and this study will not price a card on an absence.

6. Unit economics only — a deliberate constraint

Both traces span 58.9 minutes. We report cost per request, cost per 1M requests, and savings as a percentage of the priced window — never a monthly or annual figure. Scaling one hour to a month multiplies by ~720 and assumes flat demand; our own CASE-STUDY-01 measured 121 days of real traffic whose daily demand ranged from 0.7% of median to 11× median. A monthly number here would contradict the published finding sitting next to it.

The engine's canonical window is monthly, so the standard renderer prints "/mo" labels. With N set to 1,000,000 requests as the unit, every such figure reads exactly "$ per 1M requests at this mix." No annualised figure appears in this artifact or its screenshots.

7. The two findings that sized, and where they overlap

CA-01 (prompt caching) and AG-03 (duplicate calls) are the only findings this schema can size. Their engine credits at the default tier (5-minute cache (default tier), refresh):

Disclosed overlap. Input tokens belonging to exact duplicate requests (0.91% of tool/agent input tokens, 0.66% of conversation) are counted both inside CA-01's f_prefix (they are reused blocks) and inside AG-03's pct_duplicate_calls. The two findings sit in different overlap groups (input_price vs call_volume), so the engine's partition panel does not flag them. Multiplicative composition prevents additive inflation — the composed total is below the naive sum — and AG-03 is only 4.3% of the tool/agent result. We disclose it rather than leave a reader to find it.

8. Everything the audit declined to size — engine output, verbatim

45 of 54 selected findings held at $0 by the measurement invariant, each naming the variables it would need. Accounting: 54 = 2 sized + 45 unmeasured + 1 excluded (RS-02) + 6 enablers (IN-08, MS-02, QR-01, QR-02, QR-05, RS-03).

9. LRU capacity curves (at the default 5-minute tier)

tool/agent

conversation

Capacity is not the binding constraint here: 32,000 blocks = 16.4M tokens ≈ 2–5 GB of KV cache for GQA models of the relevant class — comfortably a single node. Retention, not capacity, is what sets the answer on this traffic, which is the opposite of the usual assumption.

10. What one hour of trace cannot show

• No demand shape. Commitment and utilisation findings (IN-09, IN-01/02, IN-10's time signatures) need weeks of demand curve. CASE-STUDY-01 (121 days) is the complement that measures exactly that family.
• No sessions. Agent-loop findings (AG-01/AG-02) need request grouping the release does not carry.
• No models, latency classes, quality data, or infrastructure — the MS-, RS-01, QR-, IN-* families stay unmeasured, named in section 8.
• No billing. Dollars are modeled from tokens at the cited card, never observed.
• One hour is not a TTL experiment. A 1-hour trace cannot distinguish a 1-hour cache from an unbounded one, so the 1-hour column is a lower bound on that tier's retention benefit — which makes the tier gap we report conservative.

Methodology note

Provenance. github.com/kvcache-ai/Mooncake FAST25-release/traces (Qin et al., arXiv 2407.00079; Apache-2.0). Pinned SHA-256 (the ingest refuses non-matching bytes):

• conversation_trace.jsonl — 12,031 rows — b8cbb061a85206d729d91cdc2981f43c9e0d99209dce588d3af5f7934408b9df
• toolagent_trace.jsonl — 23,608 rows — 48a2db1a13d3bc05e6330140c64f604ba366df20d3c9e128b5c35a01c1fa5f71

Excluded file: arxiv-trace/mooncake_trace.jsonl (earlier release, same row count as toolagent, different bytes — never combine).

Block semantics. hash_ids are remapped 512-token prefix-block hashes; the final block of a request may be partial (input_length ≤ 512 × len(hash_ids) is validated on every row). Timestamps are milliseconds; both spans verified ≈58.9 min.

Cache replay. Block-granular LRU, replayed in trace order, with a stated TTL and a stated capacity, touch-on-hit, insert-on-miss, lazy expiry. Each tier is priced at the write multiple that tier actually costs on the cited card — the 5-minute replay at 1.25×, the 1-hour replay at 2.00×. An untimed replay on a 59-minute trace is a 1-hour replay; pricing that at the 5-minute multiple describes a configuration no card sells, and v1.0.0 of this case study did exactly that (corrected in v1.1.0 — see changelog). Real schedulers differ in admission policy and eviction; the capacity curve and both expiry semantics are published so any operating point can be read off. The recipe is unit-tested against hand-derived fixtures including reuse just inside and just outside the window (test_mooncake_ingest.py, 50 checks).

Pricing (stated, cited, substituted openly). Kimi is a Chinese provider; its collection-era rate card is not verifiable. We price at Anthropic's published card as of 2026-07-29, re-read 2026-08-25 (platform.claude.com): Claude Sonnet 5 at $2/1M input, $10/1M output — announced as introductory pricing through 31 August 2026 and now the standard rate; the scheduled increase to $3/$15 will not occur, so the unit-cost figures in §4 do not expire. Cache read 0.1×; cache write 1.25× (5-minute) or 2.00× (1-hour) — the same r/w constants the findings library carries as vendor-verified. Moonshot's own card would scale the dollar figures; it would not change the reuse ratios or the hit-rate curves. It would change the savings percentages, as section 5 shows — which is why that section exists. The 1-hour column re-runs the same engine with w set to 2.00 in an in-memory copy of the library; findings_library.json on disk is unmodified.

Reproduce the measurements. Every step below runs on files published with this page, plus the public traces.

git clone --depth 1 --filter=blob:none --sparse https://github.com/kvcache-ai/Mooncake
cd Mooncake && git sparse-checkout set FAST25-release/traces
python3 mooncake_ingest.py FAST25-release/traces my_measurements.json
python3 test_mooncake_ingest.py
diff my_measurements.json measurements_cs03.json     # expect no output

The ingest aborts on any trace whose SHA-256 differs from the pinned hashes above, so a clean diff verifies every measured quantity on this page: both reuse definitions, the TTL-aware replay, the retention probe, the capacity curve and the duplicate shares.

What you cannot reproduce from this page, stated plainly. Converting those measurements into savings percentages runs on cost_model.py and the findings library. Neither is published — they are the audit product, not a download. build_case_study3.py and cards_cs03.py ARE published, so the card list, the crossover closed form and its assertions can be read and checked directly. And the headline arithmetic does not need the engine: CA-01's formula is printed above, so C_in · f_prefix · max(0, 1 − (h·r + (1−h)·w)) can be evaluated by hand from the measurements file for any row of the card table. The engine's contribution beyond that is multiplicative composition across findings, which moves the plan total and not the single-finding figures this study leads on.

---

Sources, every vendor figure re-read from the vendor's own live page on 2026-07-29 (v1.2.0 — the previous OpenAI row was not): Mooncake traces & paper — github.com/kvcache-ai/Mooncake, Qin et al., arXiv 2407.00079 (FAST'25), Apache-2.0. Anthropic — platform.claude.com prompt-caching documentation: "5-minute cache write tokens are 1.25 times the base input tokens price", "1-hour cache write tokens are 2 times the base input tokens price", "Cache read tokens are 0.1 times the base input tokens price", "The cache is refreshed for no additional cost each time the cached content is used"; Claude Sonnet 5 at $2/$10 per MTok — announced as introductory pricing through 2026-08-31 and confirmed the standard rate on 2026-08-25, with the scheduled $3/$15 increase cancelled, so the unit-cost figures in §4 do not expire. OpenAI — developers.openai.com prompt-caching guide: "Cache writes have no additional fee on models before the GPT-5.6 family", "On GPT-5.6 models and later model families, cache writes cost 1.25× the uncached input token rate", "A cached prefix remains eligible for reuse for at least 30 minutes", "cached prefixes generally remain active for 5 to 10 minutes of inactivity, up to a maximum of one hour". The 0.10× read multiple is taken from the cached-input-to-input ratio within each row of the developers.openai.com pricing table rather than from its absolute prices, because that page carries Standard, Batch, Flex and Priority tiers; the ratio is invariant to which tier a reader is billed on, the absolute prices are not, and this study never needs them. DeepSeek — api-docs.deepseek.com pricing page (read multiples only; no cache-write charge is documented, so no DeepSeek row is priced). CASE-STUDY-01 (this repo) for the demand-shape constraint.

Everything this page was computed from

Downloaded and run, these reproduce the figures above. The measurements file is the build's own output, not a transcription.

• mooncake_ingest.py — the ingest: schema validation, both reuse definitions, the TTL-aware LRU replay and the retention probe
• test_mooncake_ingest.py — 50 checks, including hand-derived fixtures for the replay and its TTL boundaries
• cards_cs03.py — the one card list both builders import, with the crossover closed form and its assertions
• build_case_study3.py — runs the engine and writes the report
• measurements_cs03.json — every measurement this page renders