feat(voice): Add local-first realtime voice mode

[RONAK-AI647]

Summary

Adds a local-first voice mode for Nanocoder with push-to-talk, hands-free VAD, speech transcription, text-to-speech, and interruption support.

This PR combines the six voice-mode implementation phases and includes the fixes from will's review and this fixes issue #622

Scope clarification

The Bash executor change is required for voice-mode barge-in. Hands-free voice can interrupt an active response while a Bash tool is running, so the executor now escalates from SIGTERM to SIGKILL when a process ignores cancellation. This prevents interrupted voice sessions from leaving child processes running.

Hands-free voice should not be combined with yolo or automatic tool approval in noisy environments, because unintended speech or background noise may trigger agent actions. Push-to-talk or normal approval mode is recommended for safer use.

Features

• /voice command for enabling and configuring voice mode
• Push-to-talk recording with Ctrl+T
• Hands-free voice mode using VAD
• Local Whisper STT support
• Local Piper TTS support
• Optional cloud STT/TTS for verified OpenAI providers
• Barge-in interruption during:
  • Response generation
  • Tool execution
  • Speech synthesis
  • Audio playback
• AbortSignal propagation through tool execution
• VAD worker cleanup and bounded audio buffering
• Voice status UI
• On-demand dependency installation
• SHA256 checksum verification for downloaded assets
• Architecture-aware dependency installation
• Bundled voice plugin for published builds

Review fixes included

• Rebased onto the latest main
• Fixed voice dependency installer URLs and platform handling
• Added HTTPS-only downloads and checksum validation
• Added real installer URL-path test coverage
• Fixed optional voice plugin build and distribution
• Removed the unused realtime capability abstraction
• Added Ctrl+T enabled-state protection
• Replaced repeated preference disk reads with cached reactive preferences
• Fixed VAD shutdown races and microphone orphaning
• Separated recording and TTS temporary-file ownership
• Fixed VAD temporary WAV leaks
• Added sustained-speech buffer limits
• Filtered [BLANK_AUDIO] responses
• Restricted cloud audio support to verified OpenAI providers
• Wired voiceName into cloud TTS
• Fixed signal-killed process handling
• Restored affected documentation
• Added a changeset
• Included voice plugin files in Biome validation

Validation

• Main TypeScript checks pass
• Voice plugin TypeScript checks pass
• Biome checks pass
• 36 focused voice tests pass

Notes

Native provider-specific realtime streaming is intentionally not included in this PR. Cloud voice currently uses STT/TTS REST endpoints, while all providers retain the existing Nanocoder request pipeline.

Hands-free VAD combined with automatic tool approval may require an additional maintainer decision regarding ambient-noise safety.

[github-actions]

nc-review: comments — 6 important, 2 nits

@RONAK-AI647 — a few things worth a look, none blocking.

Adds a substantial local-first voice mode (separate plugin + core wiring, push-to-talk, VAD, STT/TTS, barge-in, install queue). The PR is the cleanup pass after an earlier review, and the self-reported review-fix list is plausible and on-point. With no diff visible from the base checkout I cannot directly verify the new code paths or tests; from what I can read in base files, the wiring touches the right seams (lazy command registry, chat input, interactive-app, global handler queues, config schema, preferences, bash executor) and the plugin isolation under plugins/voice/ is the right shape. The main concerns are process: no linked issue, no prior discussion, and a drive-by bash-executor fix mixed into a feature PR.

🟠 important · warranted

This is a ~50-file feature with no linked issue. CONTRIBUTING.md asks contributors to draft a spec and discuss it in issue comments before starting work on substantial changes (**No spec? Write one** - If the issue lacks a spec, draft one and post it in the issue comments for discussion before starting work.). The project rubric says to prefer "this wants a conversation" over "this should not exist" — that is the right framing here, not a rejection. The fix is to land this behind a maintainer discussion (open an issue summarising the design and the six phases), or have a maintainer ack the PR description as that spec. Without one, a future contributor cannot tell what was already considered and rejected (e.g. the author notes "Native provider-specific realtime streaming is intentionally not included in this PR" — that decision deserves a paper trail).

🟠 important · scope

source/services/bash-executor.ts is modified for "signal-killed process handling", which is unrelated to voice. The rubric calls out "drive-by refactors mixed into a functional change — they are harder to review and harder to revert" as a scope concern. The fix either belongs in a separate PR, or — if it is genuinely needed for voice-mode behaviour — the connection has to be called out in the PR description (which it is not). Either split it, or add a one-paragraph justification in the PR body explaining how the voice path triggers a code path that the bash-executor fix unblocks.

🟠 important · completeness

The voice plugin lives under plugins/voice/, but the core wiring (lazy command registry, App.tsx, chat-input.tsx, interactive-app.tsx, useGlobalHandlerQueues.tsx, preferences.ts, types/config.ts) is modified in source/. That means the core build will not work without the plugin. The PR description says "Bundled voice plugin for published builds" but package.json's "files" list (dist, schemas, source/app/prompts/sections, source/config/themes.json, assets/nanocoder-vscode.vsix) does not include plugins/voice/dist or anything that would publish the compiled plugin. If the plugin is supposed to ship in the published package, the files array needs plugins/voice/dist (or similar) added, and the core wiring needs a runtime dynamic import of the plugin that resolves cleanly when the plugin is absent (the PR adds on-demand installation, but a fresh pnpm i followed by nanocoder may still hard-fail). Without seeing the plugin loader code I cannot tell which, but the package.json needs to be reconciled.

🟠 important · completeness

source/types/config.ts and source/config/preferences.ts are both in the changed-file list. Voice preferences become part of the public config schema (agents.config.json is explicitly called out in the project rubric as a public contract). The schema diff is not visible from the base checkout, so I cannot verify defaults are sensible, that the schema regenerator (pnpm generate:schema) is updated, or that existing user configs without the new keys still load. A maintainer reading the PR needs the full config delta visible — please confirm schemas/agents.config.schema.json is regenerated and committed, and that new keys default to off so an upgrade does not enable voice for users who did not opt in.

🟠 important · changeset

CONTRIBUTING.md requires a .changeset/*.md for every user-facing change. The file .changeset/voice-mode-review-fixes.md is in the changed-file list but is not on disk (correctly — it is added by the PR), so I cannot read its body. Two things to verify before merge: (1) the package name inside the changeset must be @nanocollective/nanocoder (the project rubric explicitly warns: "A wrong name passes the file-presence check and then breaks release-prepare on every subsequent push to main"), and (2) the changeset must be authored in the project's changelog voice ("a self-contained bullet describing user-facing impact, with attribution where relevant"), not as a release-note-style list of bullet items.

🟠 important · correctness

The PR description mentions "Hands-free VAD combined with automatic tool approval may require an additional maintainer decision regarding ambient-noise safety." That is a security-relevant flag the author raised themselves. In yolo/auto-accept mode, a stray noise (TV, conversation, a door slamming) could now become a tool invocation. The mitigation likely lives in the VAD thresholds and the install gate, but from the base checkout I cannot read them. At minimum the user-facing docs need to call out that voice mode + auto-accept is unsafe in noisy environments, and the install prompt should warn before the user enables both simultaneously. Please confirm this is documented in docs/features/ (the docs changes are in the PR) and surfaced during the install.

⚪ nit · completeness

biome.json currently excludes the entire plugins/ directory. The PR says it adds the voice plugin to Biome validation, which means it changes that exclusion. Worth confirming the resulting biome.json still excludes plugins/vscode/ (which has its own build pipeline and may not be Biome-clean) while including plugins/voice/, instead of a blanket !plugins becoming a blanket include.

⚪ nit · completeness

pnpm-lock.yaml is in the changed-file list. That is expected, but with this many files and a new plugin it is worth double-checking that pnpm install does not add any transitive dependency that pulls audio / native bindings into the main install (which would defeat the "optional plugin" framing and the build/bundle-size benchmark). The release-time benchmark measures dist/ size, so a large native dep under plugins/voice/ that ends up reachable from the main startup graph would move that number.

---

_{🔴 blocking · 🟠 a reviewer would ask for a change · ⚪ optional}

_{Automated code review — correctness, security, design, tests, plus duplicates and scope. A human still decides; this is not a substitute for review and is not exhaustive. The required status checks separately cover lint, formatting, types, unused dependencies, the test suite and the build. This bot never merges. Maintainers can rerun with /re-review.}

[will-lamerton]

Hey @RONAK-AI647 - looking great! Let me know when you'd like me to review :)

[RONAK-AI647]

@will-lamerton , There are some issues yet there , I am sorting them , this month was hectic , apologies for delay.

I would love to see a detailed review if you do , makes my debugging easier.