Conversation
🦋 Changeset detectedLatest commit: db798ea The changes in this PR will be included in the next version bump. This PR includes changesets to release 39 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
1egoman
changed the title
The agent sends each transcription two times. It sends a deprecated `Transcription` data packet. It also sends an `lk.transcription` text stream. The two copies fill the reliable data channel. Other reliable traffic becomes slow. This commit stops the legacy packet when no client needs it. Before each legacy publish, the agent examines the remote participants. If all applicable participants have client protocol 3 or higher, the agent does not send the packet. The agent always sends the text stream. Only STANDARD participants are applicable. These are the client SDK instances that users create. SIP, INGRESS, AGENT, CONNECTOR and BRIDGE participants do not show legacy transcripts. EGRESS participants are hidden and do not appear in the participant list. The agent also ignores its own avatar worker. An absent client protocol counts as 0, which means a legacy client. This decision is necessary, not defensive. protobuf-es gives the field the optional type, and in JavaScript all comparisons with `undefined` are false. Without the coalesce, a missing field reads as a modern client, and the agent stops the transcripts of a client that still needs them. The avatar worker test compares two values only when the attribute is present. An absent attribute and an absent local identity are both `undefined`. A direct comparison is true for each participant that has no such attribute. The gate is in `publishTranscription`. This is the only function that sends the packet. Both `handleCaptureText` and `handleFlush` use this function. Do not move the gate into `handleCaptureText`. Each legacy packet contains the full segment text with a stable identifier. The accumulated text must stay correct. If a legacy client joins during a segment, the next packet gives it the full segment. The agent calculates the result again for each publish. Do not put the result in a cache. `legacyStatusLogged` holds only the last status written to the log. It prevents a log message for each transcription chunk. This is a port of livekit/agents#7240 to node.
This commit adds five test helpers. It adds no tests. `createFakeRoom` makes a room stand-in. `isConnected` is true. If it were false, `publishTranscription` would never reach the local participant, and each test would pass for an incorrect reason. `fakeRemote` makes a stand-in for a remote participant. The `kind` value is always set. The real getter gives STANDARD when the field is absent, so a fake without a kind would not test its own condition. `makeLegacyOutput` makes a legacy sink with `Object.create` and sets the private fields directly. This prevents the constructor, which attaches listeners to a real room. `captureAndFlush` sends text through the sink and then waits for the flush task. `publishedSegments` collects the segments from all publish calls.
This commit adds three tests for the client protocol comparison. The first test shows that the agent sends no legacy packet when all clients have client protocol 3. The second test shows that the agent sends the legacy packets when one client is older. The test also examines the packets. The partial packet and the final packet must have the same segment identifier and the full text. The third test shows that an absent client protocol counts as a legacy client. This test has no equivalent in the Python code, because the JavaScript behavior is different. protobuf-es gives the field the optional type, and `undefined < 3` is false. Without the coalesce in the gate, this participant reads as a modern client and loses its transcripts. The failure is silent.
This commit adds nine tests. These tests hold the rules about applicable participants. Change these rules only with care. Two conditions show that the agent sends no legacy packet when the room has no STANDARD participant. An empty room and a room with only a SIP participant give the same result. A SIP participant does not show transcripts. Four conditions show that SIP, INGRESS, AGENT and CONNECTOR participants do not control the result. A further test shows that a STANDARD participant with an old client protocol does control the result. Two tests show the avatar worker rules. The agent ignores its own avatar worker. The agent does not ignore the avatar worker of a different agent. The last test has no equivalent in the Python code. It sets the local identity to `undefined`. An absent attribute and an absent local identity are both `undefined`. A direct comparison of the two is true, and the agent then ignores each participant that carries no publish-on-behalf attribute. The room becomes empty for the gate, and all legacy transcripts stop.
This commit adds two tests. The first test shows that the gate obeys changes in the room. All clients are modern at the start, and the agent sends no packet for the first text. A legacy client then joins. The agent sends one packet for the second text. This packet contains the full text of both parts, not only the second part. The final packet has the same segment identifier. This test fails if a person moves the gate into `handleCaptureText`. The second test shows that the `lk.transcription` text stream does not change. The stream always goes to all participants, also when the agent sends no legacy packet.
1egoman
force-pushed
the
port-transcription-dedup
branch
from
September 18, 2026 16:33
e68538e to
db798ea
Compare
1egoman
marked this pull request as ready for review
September 18, 2026 16:39
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
The agents-js port of livekit/agents#7240. A high level summary of the transcription deduplication project:
In livekit/client-sdk-js#2093, I added a new client protocol of 3. Any client which advertises this new client protocol version ignores all legacy transcriptions, and back-converts modern transcriptions into in-memory legacy transcriptions.
This pull request implements the converse of this behavior on the agent end: if any connected client sdk advertises support for the client protocol of 3, then the agents sdk now will skip sending legacy transcriptions to all participants.
A few important caveats:
destination_identitieswouldn't only deliver the packet to the listed participants.Pre-Review Checklist
Testing
restaurant_agent.tsandrealtime_agent.tswork properly (for major changes)In addition to newly added unit tests, I have tested this end to end locally - I have run the
client-sdk-jsboth advertising client protocol of 2 and client protocol of 3:packet.value.case === "transcription", both legacy and modern transcription events emittedNote to reviewers: Please ensure the pre-review checklist is completed before starting your review.