Skip to content

feat: add FunASR dataset annotation tool - #1309

Open
LauraGPT wants to merge 1 commit into
fishaudio:mainfrom
LauraGPT:codex/funasr-annotation
Open

feat: add FunASR dataset annotation tool#1309
LauraGPT wants to merge 1 commit into
fishaudio:mainfrom
LauraGPT:codex/funasr-annotation

Conversation

@LauraGPT

Copy link
Copy Markdown

Is this PR adding new feature or fix a BUG?

Add feature.

Is this pull request related to any issue? If yes, please link the issue.

Closes #1291.

Summary

  • add an optional FunASR dataset annotation CLI that recursively creates Fish Speech .lab files from WAV, MP3, and FLAC audio
  • preserve curated labels by default, with explicit dry-run and overwrite modes, atomic writes, single-file failure isolation, and deterministic statistics
  • keep FunASR outside the core dependency surface through lazy imports and document CUDA/CPU usage in English and Chinese

The default SenseVoiceSmall path writes plain, one-line UTF-8 transcripts for Mandarin, Cantonese, English, Japanese, and Korean. Model selection, device, language, and ITN remain configurable.

Validation

  • python3 -m unittest discover -s tests -v (9 passed)
  • python3 -m compileall -q fish_speech tools tests
  • all repository pre-commit hooks passed on the four changed files
  • dependency-free --help and --dry-run smoke passed without importing FunASR
  • real H100 smoke with FunASR 1.3.27 and the official SenseVoiceSmall Chinese and English samples: 2 processed, 0 failed; exact expected transcripts, one line each, with control tags removed
  • second-run skip and --overwrite paths both verified; clean validation environment reports no broken requirements
  • git diff --check and credential scan passed

No Fish Speech runtime dependency or training behavior changes are included.

@LauraGPT

Copy link
Copy Markdown
Author

@leng-yue, could you review this optional dataset-annotation tool when convenient? You recently merged changes in the same installation/finetuning documentation area.

The current head is CLEAN and mergeable, with no unresolved review threads or failed/pending checks. FunASR stays lazily imported outside Fish Speech's core dependencies; the CLI preserves existing .lab files by default and has dry-run, overwrite, atomic-write, and per-file failure coverage. Exact-head validation includes 9 unit tests, repository pre-commit hooks, dependency-free help/dry-run smokes, and real H100 SenseVoiceSmall transcription for the official Chinese and English samples.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature Request: Use FunASR for training data annotation

1 participant