fix(asr): keep the digit "0" in convert_num_to_words (WER groundtruth cleaning)#15888
Open
harjothkhara wants to merge 1 commit into
Open
fix(asr): keep the digit "0" in convert_num_to_words (WER groundtruth cleaning)#15888harjothkhara wants to merge 1 commit into
harjothkhara wants to merge 1 commit into
Conversation
convert_num_to_words iterated a number's digits with `while num:`, which is
skipped when num == 0, so a standalone "0" token produced no output and was
silently dropped ("the answer is 0" -> "the answer is"). Because
clean_label(..., num_to_words=True) defaults to on and is applied to the
reference text in cal_write_wer(clean_groundtruth_text=True), references
containing a bare "0" lost that word, corrupting both the reference and the
WER denominator.
Emit "zero" explicitly for the num == 0 case; all other inputs are unchanged.
Adds a GPU-free unit test covering the standalone "0" and non-regressions.
Signed-off-by: harjoth <harjoth.khara@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do ?
Fixes a silent word-drop in
convert_num_to_words(): a standalone"0"token is deleted instead of becoming"zero", which corrupts groundtruth text (and the WER denominator) during evaluation.Collection: ASR (
nemo/collections/asr/parts/utils/eval_utils.py)Changelog
convert_num_to_wordsbuilds each number digit-by-digit withwhile num:. When a token is the bare string"0",num = 0is falsy, so the loop body never runs and the token produces no output — it is silently dropped. Every other digit works, and"0"embedded in a larger number ("10","100") works because the loop is entered by the nonzero leading digit.clean_label(_str, num_to_words=True, ...)defaultsnum_to_words=Trueand is applied to the reference incal_write_wer(..., clean_groundtruth_text=True). So a reference like"the answer is 0"becomes"the answer is"— the reference loses a word, changing both its content and the WER/CER denominator."zero"explicitly for thenum == 0case; all other inputs are unchanged (minimal, behavior-preserving). Note: leading-zero handling ("007"->"seven") is a separate pre-existing quirk and is intentionally left untouched to keep this change single-purpose.Usage
Before your PR is "Ready for review"
Pre checks:
tests/collections/asr/utils/test_eval_utils.pyPR Type:
Additional Information