offload-gain: savings math prices every re-send at full input — with prompt caching held for N turns it overstates ~5.6x

[roy-tong]

Tested at commit 9b3aacc (main, 2026-09-09), scripts/offload-gain + README "The money" section.

What the report claims

would_have prices the context the main model would have held as full input rate × amp turns:

would_have = (tin + b_tok_undelegated) * main_in * a.amp
still_pay = tout * main_in * a.amp
usd = would_have - worker_cost - still_pay

The README's worked example spells out the same arithmetic: Opus holds a 20K-token file for 10 turns → $1.000, delegated to Sonnet costs $0.070, "that is the 93%".

Why that overstates in the tool's own target environment

Claude Code (the host this hooks into) keeps the conversation prefix in the prompt cache: the first send is a cache write (1.25x input) and every re-send is a cache read (0.1x input — e.g. models.dev lists Opus 4.5/5 at $5 input / $0.50 cache read). So holding 20K tokens for 10 turns actually costs:

• write: 20,000 × $6.25/1M = $0.125
• 9 re-reads: 9 × 20,000 × $0.50/1M = $0.090
• total $0.215, not $1.000

Net saving becomes 0.215 − 0.045 (worker) − 0.00225 (the 500-token answer re-read) ≈ $0.165, vs the reported $0.930 — about 5.6x overstated, and the headline "93% saved" is ~78%. The worker side is billed at full input rate (conservative), but it's an order of magnitude smaller than would_have, so it doesn't offset this.

The flat-rate model is the honest one only when consecutive turns are more than a cache TTL apart (5m), so the prefix really is re-bought every turn — worth saying, since that's when offloading helps most.

Reproduction (real script, synthetic ledger row matching offload_ledger's schema in scripts/lib/common.sh):

$ OFFLOAD_LEDGER=/tmp/t/ledger.jsonl python3 scripts/offload-gain --rate 5 --amp 10 -v
Reading these files yourself would cost   $   1.0000
What the cheap worker charged instead    -$   0.0450
What you still pay for its answers       -$   0.0250
You saved                                 $   0.9300

Suggested fix

Keep the flat figure (right when the cache expires between turns) and print the cache-aware one beside it. Patch (14 lines, verified against the repro above):

# prompt-cache-aware companion: first send a cache WRITE (1.25x input),
# every re-send a READ (0.1x input)
hold_mult = 1.25 + 0.10 * (a.amp - 1) if a.amp >= 1 else a.amp
would_have_cached = (tin + b_tok_undelegated) * main_in * hold_mult
still_pay_cached = tout * main_in * hold_mult
usd_cached = would_have_cached - worker_cost - still_pay_cached

plus a With prompt caching line in the -v money block, usd_saved_cached in --json, and a "Prompt caching" bullet in the README's "What the number does not include" (it currently lists follow-up verification, bytes/4 and media, but not this). With the patch the repro prints:

  You saved                                 $   0.9300
  With prompt caching (1.25x write, 0.1x re-reads)
    would have paid / saved                 $   0.2150 / $0.1646

Happy to turn this into a PR if the approach sounds right. The fixture I used is one synthetic ledger line — no external tooling needed to check it.