fix(xai): reconnect the STT stream when its socket drops - #2538
Open
CodeWithMoin wants to merge 2 commits into
Open
CodeWithMoin wants to merge 2 commits into
CodeWithMoin wants to merge 2 commits into
Conversation
Promise.all was handed the wsMonitor Task itself, which is not thenable, so it resolved instantly and an unexpected close never reached the retry loop: the stream just stopped recognizing. Await wsMonitor.result instead, the same change livekit#2470 made for Deepgram. That makes the retry reachable, so also scope each attempt's audio sender to the connection: cancel its queue read and wait for it to settle before reconnecting, so a leftover sender cannot steal frames from the new socket. Refs livekit#2469
🦋 Changeset detectedLatest commit: 32ecb6b The changes in this PR will be included in the next version bump. This PR includes changesets to release 39 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
listenTask runs on the stream-wide abortController, so a closed socket settled neither branch of its race and the old message handler stayed attached after a reconnect, still able to feed late events into the stream. Race it against the attempt signal too, detach the handler when it ends, and wait for it in teardown alongside the sender.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
The xAI half of #2469. When the xAI STT socket drops mid-session, the stream never reconnects, it just stops recognizing. Same root cause #2470 fixed for Deepgram:
Promise.allgot thewsMonitorTask itself, which isn't thenable, so it resolved instantly and the monitor's rejection never reached the retry loop.Switching to
wsMonitor.resultmakes the retry reachable, which exposes a second problem, so this also scopes each attempt's audio sender to its connection. Otherwise the old sender keeps pulling frames off the input queue after a reconnect and steals them from the new socket.Changes Made
plugins/xai/src/stt.ts:wsMonitor.resultin#runWSattemptAbortController per connection: the sender readsinput.next({ signal })so the parked read is cancelled, not just raced, and#runWSaborts it and waits for the sender to settle before returningtranscript.createdalso ends on abort, so a socket that drops before the server is ready can't hang teardownaudio.doneonly goes out on an open socketplugins/xai/src/stt_reconnect.test.ts: new test with a localwsserver that drops the first connection as soon as audio arrives, then checks the stream reconnects and gets a final transcript from the second one..changeset/xai-stt-reconnect.md: patch for@livekit/agents-plugin-xai.Same shape as the Deepgram fix, minus the heartbeat. Happy to add that too if you want parity.
Pre-Review Checklist
Testing
restaurant_agent.tsandrealtime_agent.tswork properly (for major changes): not a major changeThe new test passes 3/3 with the fix. On
mainit times out, since the stream never reconnects.Additional Notes
Refs #2469. Deepgram was the other half, already merged in #2470.