Enable web-based LoRA/full-model training with live output#1
Enable web-based LoRA/full-model training with live output#1KhryptorGraphics wants to merge 17 commits into
Conversation
… local artifacts - analyze_spectrum(.py/2.py): spectral comparison diagnostics used to chase the fast-note-change 'autotune' artifact and detection-percentage tuning - create_svcfork_checkpoint.py: build svc-fork checkpoint metadata - update_connor_profile.py: profile json updater for the Conor voice - .gitignore: keep local debug launchers (inline flask secret), ruvector.db, .codegraph/, playwright live reports, and .claude/settings.local.json out of the repo
54 agent definitions, command templates, and hook helpers referenced by CLAUDE.md so the tooling setup is reproducible across checkouts. Machine-local settings (.claude/settings.local.json) stay untracked.
|
Seems you are using me but didn't get OPENAI_API_KEY seted in Variables/Secrets for this repo. you could follow readme for more information |
|
Too many files changed for review. ( Bypass the limit by tagging |
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
…c device; add local-audio ingestion + V100 launchers - trainer: DataLoader uses spawn context (16 workers) to avoid fork+CUDA deadlock after GPU ContentVec precompute; warmup scales from immutable scheduler base_lrs (was get_last_lr(), which compounds ExponentialLR and collapsed lr to ~0 on small datasets) - encoder: ContentVecEncoder.encode aligns the HF model to the input device (latent CPU/CUDA mismatch surfaced by multi-sample on-the-fly extraction) - scripts/ingest_local_audio_profile.py: reusable video/audio -> profile ingestion via pretrained htdemucs_ft, optional --slice-seconds clipping - scripts/serve_local_threading.py: threading-mode server (no eventlet) - scripts/start_v100.sh: V100 launcher, UUID GPU pin + sm70 assertion
|
Seems you are using me but didn't get OPENAI_API_KEY seted in Variables/Secrets for this repo. you could follow readme for more information |
- serialize concurrent log fetches (in-flight guard) so the immediate poll + interval + React StrictMode double-invoke can't all fetch at offset 0 and append the first lines twice - discard a log response whose jobId switched mid-flight (cancel -> start new run) so the old job's lines don't append into the new job's log
|
Seems you are using me but didn't get OPENAI_API_KEY seted in Variables/Secrets for this repo. you could follow readme for more information |
…f16 gate - voice_profiles.list_training_samples re-anchors stored absolute sample paths to the store's actual location, so profiles survive data-dir relocation / cross-machine sync (AV-ksek) - VocalSeparator.separate clamps a too-large segment to the model's max (min across sub-models for htdemucs_ft), preventing the >7.8s htdemucs reshape crash for every caller; api_profiles no longer passes 10.0 (AV-e4p7) - trainer.resolve_precision downgrades bf16 -> fp16 on pre-Ampere GPUs (capability < sm80, e.g. V100 sm70), fail-safe on unknown capability (AV-ua0w) - add tests/test_bugfix_paths_segment_precision.py
|
Seems you are using me but didn't get OPENAI_API_KEY seted in Variables/Secrets for this repo. you could follow readme for more information |
ConversionHistoryPage crashed with 'Cannot read properties of null (reading toFixed)' when a conversion record had rtf/processing_time_seconds === null (the !== undefined guard let null through). Guard with typeof === 'number' so the history page renders records missing those metrics.
|
Seems you are using me but didn't get OPENAI_API_KEY seted in Variables/Secrets for this repo. you could follow readme for more information |
|
Seems you are using me but didn't get OPENAI_API_KEY seted in Variables/Secrets for this repo. you could follow readme for more information |
…training UI Fixes crash 'Cannot read properties of undefined (reading toFixed)' on checkpoint.final_loss (LoRA-only checkpoints omit web metrics). Swept the same class on the training/profile/history path: TrainingJobQueue (null-passing !== undefined guard), GPUMonitor, VoiceProfilePage/ConversionWorkflowPage sample durations, and QualityMetricsPanel gauges. QualityMetrics leaves the async metrics path (get_job_metrics -> _calculate_metrics) omits — mean_error_cents, embedding_distance, spectral_distortion — marked optional; GaugeProps.value widened + NaN bar-width clamped.
|
Seems you are using me but didn't get OPENAI_API_KEY seted in Variables/Secrets for this repo. you could follow readme for more information |
…); file 3 follow-ups (api.ts split, ConversionHistoryPage guard, fp32 mem watch)
|
Seems you are using me but didn't get OPENAI_API_KEY seted in Variables/Secrets for this repo. you could follow readme for more information |
…al-decoder <-> svc-fork-integration divergence)
|
Seems you are using me but didn't get OPENAI_API_KEY seted in Variables/Secrets for this repo. you could follow readme for more information |
…d, fp32 mem); record AV-owvr divergence assessment
|
Seems you are using me but didn't get OPENAI_API_KEY seted in Variables/Secrets for this repo. you could follow readme for more information |
…n unique commits cherry-picked into hq-lora-universal-decoder)
|
Seems you are using me but didn't get OPENAI_API_KEY seted in Variables/Secrets for this repo. you could follow readme for more information |
|
Seems you are using me but didn't get OPENAI_API_KEY seted in Variables/Secrets for this repo. you could follow readme for more information |
|
Seems you are using me but didn't get OPENAI_API_KEY seted in Variables/Secrets for this repo. you could follow readme for more information |
1 participant
Summary
training_mode(lora|full) andarchitecture(como|diffusion_mel|mel_gan|svc_fork).force: trueoverride.TrainingJobManager._save_trained_adapter()now returns/persistsadapter_model/serving_model_path, and_update_profile_training_state()prefers the self-contained serving artifact over the deltas-only adapter file.GET /api/v1/training/jobs/<job_id>/logs?offset=Nendpoint backed byTrainingJobManager.append_job_log()/get_job_logs().TrainingConfigPanelexposes LoRA vs Full-model mode and the CoMoSVC architecture;ConversionWorkflowPagesendsforce: truefor early full-model starts and surfaces returned warnings;LiveTrainingMonitorpolls the new logs endpoint and renders a live output panel.Verification
python -m py_compile src/auto_voice/web/api_training.py src/auto_voice/training/job_manager.py src/auto_voice/cli.py— clean.pytest -q tests/test_training_ui_routes.py— 9 passed (4 new: config-options architecture, unknown-QA warning path, force-override full training, job logs buffering).cd frontend && npm run build— clean production build.cd frontend && npm run test:e2e -- training-ui.smoke.spec.ts— 3 passed.POST /training/jobs(lora, force=true) -> 201;GET .../logs-> real buffered lines;POST .../cancel-> 200.detect_changestimed out on this box;code-review-graphequivalent used as the scoped fallback).Commits
3564457cEnable web training mode selection331b1027Refresh agent helper metadata (unrelated.claude/helpers/*+agentdb.rvf*refresh, split out per session notes)