Skip to content

Fix CosyVoice2 CUDA synthesis and integrate root dependencies - #150

Open
sunxfancy wants to merge 3 commits into
devfrom
codex/cosyvoice2-cuda
Open

sunxfancy wants to merge 3 commits into
devfrom
codex/cosyvoice2-cuda

Conversation

@sunxfancy

@sunxfancy sunxfancy commented Nov 11, 2026 •

Copy link
Copy Markdown
Owner

CosyVoice2 could not complete real SSUI synthesis because CUDA speech dependencies were absent from the root runtime, reference tensors were used as booleans, and the adapter discarded all but the first audio chunk.

This change declares the Linux/Windows x64 CUDA inference dependencies in dependencies/dependencies.yaml, regenerates their requirements/locks, preloads ONNX CUDA libraries, and returns complete WAV-backed Voice results. It preserves path-backed prompts and the existing opt-in quantization/CPU-offload controls.

Memory use is reduced with lazy, cached language normalizers, low-memory Qwen initialization, and memory-mapped checkpoint loading. yarn test:cosyvoice uses pre-normalized text by default; full text normalization is explicitly enabled with --text-frontend. Real-model tests run in a fresh interpreter and retain audio and device reports.

Validation:

  • yarn requirements:check: passed.
  • yarn test:dependency-config: 15 passed, including shared WAV dependencies on every supported profile.
  • yarn test_on cosyvoice_adapter_test: 5 passed.
  • Real CUDA synthesis on the rebased PR source: RTX 3090, PyTorch 2.4.1+cu124; LLM/Flow/HiFT on cuda:0; ONNX CUDAExecutionProvider active; generated 2.64 seconds of 24 kHz audio.
  • GPU validation ran serially in a systemd scope with MemoryMax=4 GiB and MemorySwapMax=0; exit 0 and zero swaps. OS-reported maximum RSS was 4,377,880 KiB, which includes shared mappings and differs from cgroup charging.
  • Targeted public-loader memory-controls regression: passed.
  • git diff --check: passed.

The GPU probe reused the existing Python environment and model cache. A fresh local install of the complete latest-dev dependency set, Windows inference, a full local regression run, and first-time FST compilation were not performed. The hosted macOS build subsequently passed all 393 tests (30 skipped under existing conditions). Other runtime profiles retain upstream constraints.

CI follow-up: SoundFile is shared across runtime profiles; the hosted macOS build passed 393 tests (30 existing skips). Linux jobs that install system dependencies now use Ubuntu 24.04, matching release builds. A CI-only APT helper retains Ubuntu signature verification, suites and third-party repositories, uses HTTPS mirrors with independent fallback, requires complete index updates, and bounds each attempt to 120 seconds. APT failures stop package installation instead of reusing stale indexes. All browser/native test gates remain enabled. Eight focused APT tests and YAML parsing passed locally; hosted validation of commit 66830d4 is in progress.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant