Per-turn smart routing with decision-aware tiered downgrades

[commoddity]

Functional lines changed: 7 files, +732 −325

Key files to review

• internal/gateway/router.go — per-turn routing engine with 3-tier override system, turn-type detection, tool verbosity cap
• internal/usage/query.go — query layer refactor to generic queryAndScan/scanRows helpers and shared SQL constants
• internal/gateway/proxy.go — cross-provider model re-resolution after override; upstream error passthrough improvements
• internal/gateway/auth.go — refactored from per-handler auth to AuthMiddleware middleware with constant-time comparison
• internal/gateway/server.go — wired auth middleware into route registration

Summary

• Per-turn smart routing: Classifies each agent-loop turn (tool result, user prompt, agent continue) and downgrades cheap turns to flash-tier or pro-tier models based on turn type, tool output size, and tool risk profile (read-only vs write/decision)
• Decision-aware medium tier: Read-only tool results (Read, Grep, Glob, WebSearch, WebFetch, FetchMcpResource) route to flash; write/decision tool results with non-trivial output route to a mid-tier pro model instead of all-or-nothing downgrade
• Tool verbosity cap: Flash-tier turns get max_tokens=1500 to prevent cheap model turns from ballooning into multi-thousand-token streams that stall the agent loop
• Auth middleware refactor: Replaced inline requireGatewayKey calls with a composable AuthMiddleware wrapping route handlers; added constant-time gateway key comparison and case-insensitive Bearer scheme parsing
• Override cross-provider re-resolution: When a model override maps to a different provider (e.g. kimi-k3 → deepseek-v4-flash), the proxy now re-resolves the provider from the overridden model to ensure requests reach the correct upstream
• Upstream error passthrough: Non-JSON upstream error bodies are now wrapped with the raw body text instead of a generic placeholder, so Cursor sees the actual provider error message

Non-functional changes

• .cursor/rules/gateway.mdc, deepseek.mdc, release.mdc — updated router and tunnel docs for per-turn routing behavior
• .cursor/skills/open-pr/SKILL.md — new skill for standardized PR creation in this repo
• README.md — expanded architecture and routing documentation