Repeat Pad vs Zero Pad for training speaker label models

[CrypticFate5]

https://github.com/NVIDIA-NeMo/NeMo/blob/75ee749762b97f41e8174c521d317f3dcff04052/nemo/collections/asr/models/label_models.py#L267

https://github.com/NVIDIA-NeMo/NeMo/blob/75ee749762b97f41e8174c521d317f3dcff04052/nemo/collections/asr/data/audio_to_label.py#L460

Why is that we use repeat pad for training speaker label models, why not just use zero pad ? My training experiments (trained for LID task) also align with this fact that repeat pad is better, but do we have any better explaination as to why this was choosen?

One reason I can map to is that the say I'm using some convolution involved encoder, so padding can accumulate over the layers and pollute the speech which has shorter length from the backwards with padding tokens influence. But I'm not sure as how same encoders like FastConformer are used ASR models, which works well for shorter audios as well. Shouldn't this padding poison problem occur there too?

Main reason for asking is, how do we account for batches which have varying lengths audios and there must be padding done there to feed the model during inference ? Say I have a batch size of 4 and the audios in the batch are of length 1s, 5s, 20s and 25s, so in repeat pad we'd be repeating the 1s audio 25 times and the audio length also becomes 25s now and in zero pad 1s audio would be padded to the 25s with zeros and we'll maintain the audio length same as 1s.

Is this even a correct way of doing it? Please share your thoughts on this.

@nithinraok

[nithinraok]

For speaker tasks, we enforce a minimum duration (e.g., ≥3s) using repeat padding to keep inputs speech-like and avoid introducing artificial silence. This stabilized training and keeps feature statistics consistent. Repeated segments actually made learning easier since the model focuses on speaker characteristics, not temporal content alignment.

For ASR, the objective depends on precise time alignment between audio and tokens, so repeating audio would duplicate sequences and corrupt the targets. Hence zero padding with masking is used instead there.

[CrypticFate5]

Hi @nithinraok , thanks again for the explanation.

I still have a few follow-ups from recent ablations on my LID setup:

1. You mentioned that

stabilized training and keeps feature statistics consistent

In practice, my train batches contain mixed durations (roughly 2–30s), so repeat-padding can expand short clips to the longest sample in-batch. At inference, batch composition varies a lot (all-short / mixed / all-long), and I’m seeing measurable batch-sensitivity: predictions differ between repeat-pad and zero-pad evaluation, with zero-pad often slightly better.

2. Could you clarify the intended semantics for speaker/LID inference:

• repeat-pad + original true lengths (a), or
• repeat-pad with padded lengths (b), or
• zero-pad + true lengths (c) ?

Note: b and c produced me same benchmark results, might confirm that there is no padding leak issue due to convolutions.

I want to ensure I’m matching the expected behavior exactly.

3. For very short LID cues (<3s, especially ~1s), what has worked best in your experience?

I tried chunk-based augmentation (multiple chunks from same utterance in-batch) and only got limited improvement.
I’ve run controlled ablations where only collate/padding behavior was changed, and the trends are consistent in thetest sets, so I’d like to align with best practices.

Thanks, any concrete recipe for short-utterance LID would be extremely useful.

[CrypticFate5]

Hi @nithinraok , any insights on this ?