Skip to content

Harden message delivery in CLI and serve/plugin mode against transient failures #80

Description

@FernandoCelmer

Context

Follow-up to #79 (assistant response lost on mid-stream timeout, already fixed). Further investigation found more places where a message/turn can be silently lost in the terminal (pycodeloop/cli/chat.py) and in serve/plugin mode (pycodeloop/cli/serve.py), plus gaps in the original fix's exception coverage.

Findings

Same class as #79 (incomplete partial-preservation)

  1. pycodeloop/providers/generic.py _stream() — the except clause only covers TimeoutError, ConnectionError, urllib.error.URLError. Other exceptions raised mid-read (ssl.SSLError, http.client.IncompleteRead, a malformed chunk causing KeyError/TypeError) still propagate raw and discard the accumulated text/pending.
  2. pycodeloop/cli/chat.py_text_buffer (filled by _on_text_delta) is only flushed/shown on the success path (_on_usage, called after agent.run() returns normally). _run_turn's generic except Exception logs a bare error and never surfaces _text_buffer, so any exception not covered by fix Per-role model fallback chains in Agent #1 above still wipes out visibly-streamed text from the terminal.

Structural (serve.py / plugin protocol)

  1. pycodeloop/cli/serve.py — malformed NDJSON input line is silently continued with no log/response, leaving the client waiting forever on that request id.
  2. pycodeloop/cli/serve.py protocol has no heartbeat/keepalive. During long reasoning or a long-running tool with no incremental output, a client (e.g. a VSCode extension) with its own read timeout may conclude the process died and drop the connection.
  3. pycodeloop/cli/serve.py _send() has no try/except around sys.stdout.write/flush. If the client already disconnected, BrokenPipeError propagates through any on_* callback, _respond_error then tries to send again and raises a second unhandled BrokenPipeError, killing the daemon thread silently with no actionable log.
  4. pycodeloop/cli/serve.py confirm() blocks on answer_queue.get() with no timeout (unlike chat.py's CONFIRM_TIMEOUT = 3.0 auto-approve). A disconnected/hung client deadlocks that chat thread forever.
  5. pycodeloop/core/agent.py (on_message, on_tool_call, on_tool_result, on_usage, on_context, etc.) and pycodeloop/core/codeloop.py's on_message = lambda: storage.post(...) — no callback is wrapped in try/except. A transient storage failure (disk error, DB down) aborts the entire in-flight turn instead of just failing that one persistence attempt.

Proposed fixes

  1. Widen the _stream() except clause to catch any exception once something has already been accumulated (text or pending), always preserving partial output instead of an exception whitelist.
  2. In chat.py, flush _text_buffer into the log (as an interrupted/partial response) in _run_turn's error handler before showing the generic error.
  3. In serve.py, log malformed input lines and send an error response referencing the request id when parseable, instead of silently dropping them.
  4. Add a periodic heartbeat/keepalive notification in serve.py while a turn is in flight (e.g. during long provider or tool calls) so the client knows the process is alive.
  5. Wrap _send() in try/except for BrokenPipeError/OSError, treating a broken pipe as "client gone" (stop the loop / mark disconnected) instead of raising again from the error path.
  6. Add a timeout to serve.py's confirm() with a safe default behavior (mirroring chat.py's auto-approve-on-timeout, or auto-decline if that's safer for the non-interactive plugin context).
  7. Wrap each Agent callback invocation (_notify_message and the other on_* call sites) in try/except, logging/tracing the failure without aborting the rest of the turn.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions