Raise the default context window from 4096 to 8192

[dovvnloading]

The report

Chats appeared to lose continuity after a few turns. Refined on testing to: "larger system prompts seem to be overloading the model and making it forget."

That diagnosis was right, but not for the reason it looks like — this has nothing to do with model size or attention quality.

What I checked first

Drove a real two-turn conversation through the actual /api/v1/generations → jobs → GenerationService orchestration. Turn 2 genuinely received turn 1's full persisted history. Persistence and loading are not the problem.

What's actually happening

SynthesisAgent.fit_history_to_context is working exactly as designed, against a budget that's too small. num_ctx is one Cortex setting applied identically regardless of which model is selected — it isn't tied to the model's own native context.

Measured the real trimmer (not the test double, which skips trimming and would have hidden this) against a realistic 30-exchange conversation at the old default of 4096:

condition	exchanges kept / 30
memory on, code-exec off	10
memory on, code-exec eligible	6
+ a handful of stored memories	4

The built-in system_prompt.txt + memory_prompt.txt cost ~1332 tokens before a single word of the conversation is counted. Add the code-execution JIT prompt (triggered by ordinary phrases like "run this") and it climbs to ~1973 — out of a 3072-token prompt budget (num_ctx minus the 1024-token output reservation). Every model actually installed here natively supports far more (qwen3:8b alone is 40960), so history was being discarded well within what the model could use, and nothing in the UI said so.

8192 was verified against the worst realistic combination — memory + stored facts + code-execution eligibility + custom system instructions — and keeps the full 30-exchange conversation.

Left the existing 16384 ceiling alone. Raising it trades directly against local RAM/VRAM for the KV cache — a real product decision, not a bug fix, and not what was asked for here.

Changes

Every place a copy of the old default lived:

• GenerationSettings.num_ctx (the real default)
• two defensive fallbacks in services/generation.py / services/llm.py (only hit when model_options lacks the key, which a real snapshot never does — but now stay in step with the real default)
• three frontend fallback constants shown before Settings loads
• contracts/openapi.json regenerated
• the one test that asserted the old default value

New regression test

test_default_context_window_survives_a_realistic_long_conversation reads the default from CortexSettings() rather than hardcoding it, so it stays meaningful if the default ever changes again, and fails with the exact surviving-exchange count if it regresses.

Verification

./scripts/check.ps1 — 6/6 green. 507 backend tests (was 506), 163 frontend.

🤖 Generated with Claude Code