fix: streaming timeout retry + progress keepalive during long inference

[gdeyoung]

Fix: Streaming timeout retry + progress keepalive during long inference

Problem

During long LLM inference (30-90s for 500K+ token contexts), two bugs cause session death:

1. Transient timeout classification: httpx.ReadTimeout (originating from aiohttp.SocketTimeoutError via LiteLLM's transport) is not classified as transient in _is_transient_litellm_error(), so streaming timeouts are never retried.

2. Partial-chunk guard blocks retries: The got_any_chunk guard blocks ALL retries once any streaming data is received, even for transient errors.

Fix

Part 1: Transient classification (models.py)

Add httpx timeout exceptions to the transient_types tuple:

transient_types = (
    ...existing types...,
    # Streaming socket timeouts — aiohttp.SocketTimeoutError maps to httpx.ReadTimeout
    httpx.ReadTimeout,
    httpx.ConnectTimeout,
    httpx.PoolTimeout,
)

Part 2: Retry condition (models.py)

Allow retries for transient errors even when partial chunks were received:

# Before: if got_any_chunk or not _is_transient_litellm_error(e) or attempt >= max_retries:
is_transient = _is_transient_litellm_error(e)
if attempt >= max_retries:
    raise
if not is_transient and got_any_chunk:
    raise

Part 3: Progress keepalive (agent.py)

Wrap call_chat_model with a background heartbeat task that emits progress events every 10s during long inference, keeping the Socket.IO connection warm:

_stop = asyncio.Event()
async def _heartbeat():
    count = 0
    while not _stop.is_set():
        try:
            await asyncio.wait_for(_stop.wait(), timeout=10.0)
        except asyncio.TimeoutError:
            count += 1
            self.context.log.set_progress(f"Generating... ({count * 10}s)")
_task = asyncio.create_task(_heartbeat())
try:
    response = await model.unified_call(...)
finally:
    _stop.set()
    _task.cancel()

Testing

• Verified on v2.2 with 500K+ token contexts
• Both streaming retry locations patched (unified_call + unified_turn)
• Heartbeat fires every 10s, shows elapsed time in UI progress bar