Are there better wakeword models #288
Replies: 7 comments
-
|
FYI, since this project supports openWakeWord (OWW) now you are probably best of looking for existing openWakeWord models or perhaps train your own openWakeWord model. See: and Anyway, you would be much better of asking these type of questions in openWakeWord communities instead if you want answers:
PS: For reference maybe also read this old original thread: |
Beta Was this translation helpful? Give feedback.
-
|
I am taking about both MicroWakeWord and OpenWakeWord where admittedly OpenWakeWord is the better but both strangely seem to perform much lower than benchmarks of this type of model. Still those models are extremely bloated in parameters and why I am asking if there are better is because quite a number already exist in opensource and if this is a 'Linux voice assistant' if there are better opensource wakeword models available especially streaming models that can be used to capture aligned wakeword. There are better models and better training methods and not a single thing you posted has any bearing on what is posted in the OP. |
Beta Was this translation helpful? Give feedback.
-
|
Have a look at this repo for training a custom microwakeword (very easily…even allows you to upload your own voice samples it incorporates into the training data): https://github.com/TaterTotterson/microWakeWord-Trainer-Nvidia-Docker It has basically entirely solved my false wakeword detections. I wrote up a lot more detail here: https://community.home-assistant.io/t/my-journey-to-a-reliable-and-enjoyable-locally-hosted-voice-assistant/944860/63?u=fantasymaster85 But I’m using it with LVA with great success. |
Beta Was this translation helpful? Give feedback.
-
|
That looks really good, but likely its the 'your voice' that is saving the dataset as dunno why OHV will only use piper when it just gives so little prosody change for the US English voices it provides that are a little unnaturally fast. Also in the repo notebook they used prerecorded rir all @ 1.5m of some pretty weird places such as cathedrals, forrests and stuff which was just really curious. Looking at your stuff its looks like a massive improvement and someone who understands dataset requirements. I had a look at microwakeword again as after seeing that original training notebook gave up on it, it is a true streaming model and its the averaging function that uses a rolling window. Its always puzzled me why it can not capture wakeword as spoken and create further data for a local automated finetuning, but for some reason tflites are always shared but not the full tensorflow model so you can just fine tune... As said looks good in fact amazing compared to that training script OHV has been presenting the community! |
Beta Was this translation helpful? Give feedback.
-
|
If you want a quick(ish) solution, this Python notebook can be used on a free colab T4 instance to autotrain a wakeword: https://colab.research.google.com/drive/1q1oe2zOyZp7UsB3jJiQ1IFn8z5YfjwEb?usp=sharing I used it to create mine and it did great. Outputs are tflite/onnx that work with openWakeWord type. |
Beta Was this translation helpful? Give feedback.
-
|
I have added some alternatives as examples. A CRNN streaming model from https://github.com/rolyantrauts/dataset-creation/tree/main/kws-streaming Main difference is the noise class can be used for VAD purposes so no need of Silero. https://github.com/rolyantrauts/bcresnet/tree/main/models same dataset but non-streaming bcresnet model. Its a Hey-Jarvis using all TTS output with no augmentation than noise being added with no RIR. |
Beta Was this translation helpful? Give feedback.
-
|
Nanowakeword 😎 |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
I created a repo dataset-creation with some toy examples.
1
1st you can use TTS to create datasets but using multiple TTS to garner prosody variation and any signatures a single TTS can give.
It demonstrates picking words for unknown by the phoneme of syllables which is the manner we stress language.
Also it expands by using extra classes of like sounding words to the wakeword and organised by syllable count classes to force training that way.
In a classification model you really want the same voice to have entries in all classes so that features are picked on the difference of the word and not voice spectra. That just collecting wakeword and not the large qty of 'unknown' words will always be less than optimal.
Because of the large datasets of many voices of many words doing this with synthetic TTS is easy compared to trying to get data collection organised.
Also though submitted capture is a total unknown of SNR (noise), RIR distance from mic that 'just' collecting wakeword lacks the rigorous quality control and meta-data required to create balanced quality datasets.
Its very easy to augment and add noise and reverberation via multiple tools but extremely hard to measure or remove from submitted samples.
So because you can get clean samples through TTS they are actually better than much of what is being collected.
When using common input at 16khz with quantised spectrogram such as MFCC the quality of modern TTS is indistinguishable from human voice.
There is a toy dataset that you can download in the above to save some work to experiment with linked in the readme.
Created by three opensource TTS with a dataset size of a size several orders of magnitude than the models parameters which is a good rule of thumb.
2
2nd The implementation always seems to make the presumption that its a lone authoritative wake word, whilst the 1st wakeword can be a low compute, energy saving switch purely to activate an authoritative wakeword of higher compute and greater accuracy.
I have included what is supposedly a SotA model bcresnetBSD-3-Clause-Clear license
Its been hacked out to use any dataset than the repo which is hardcoded to use the Google command dataset.
Also kws_streaming.
I will throw up some trained examples of the base dataset with no noise or rirs purely as a datum for benchmarks.
Also the same model with but with increased layer sizes and subsequent increases in parameters to test differences in accuracy.
3
3rd On device capture where streaming wakeword are far more accurate in detecting the wakeword time domain. Streaming wakeword only process the often 20ms chunk provided. A streaming wakeword is one of the most accurate forced aligners you can get and is a great way to accurately extract and store spoken wakeword. This means your wakeword can become a learning device and learn the room and users of use. Local capture and on device training can massively increase accuracy and we have the opensource tools to have total control of local data, assistants that can train but stay mostly idle and the opensource tools to do so.
4
4th Stop black-boxing an audio pipeline when linux has super efficient optimised frameworks with a huge array of plugins available. Take WebRTC AGC2 which is not designed for far-field wakeword solutions and I don't use because I have always been confused how to ensure I get the levels I have set in the wakeword dataset and not what Google might think applicable for a on-device microphone. I get better results with vlevel because I can set certain params, I know what the output is because its a ALSA plugin and can just record with standard ALSA tools
arecordand analyse.I am not sure WebRTC makes a good AGC for wakeword as like Grok I get better results using other methods.
Grok:"For ASR-focused applications, consider starting with WebRTC AGC2 if your setup involves real-time WebRTC streams, as it's readily available and tunable (e.g., via modes like adaptive digital). But for higher precision, especially in offline processing or wake-word systems, integrate or test instantaneous or MOST methods, which address ASR's sensitivity to level inconsistencies more directly. Empirical testing on your dataset is recommended, as performance varies with noise, language, and ASR model (e.g., end-to-end DL architectures)."
A voice assistant to be optimal is a tuned end-to-end DL architectures, where each process is trained to be optimal for the previous, that needs careful consideration of the audio processing for the tasks needed. If you just throw together general purpose modules it will be less optimal than a tuned system and then embedding as a black box just causes a barrier for others to learn what a single dev has done than use the widely documented Linux tools that already exist...?!?
This is a strange one as a voice-assistant is a pipeline of models that each model should be finetuned to the output of the previous model/process and for some reason it has never been done.
Your wakeword should be trained by augmenting the dataset with the speech-enhancement/AGC of use, AGC should come after speech-enhancement, but because it does it doesn't need AGC2 types.
This is the same for ASR as ASR not trained for speech enhancement but raw noisy input will perform less optimally with speech enhancement unless the dataset is augment with the previous models/modules of use, or finetrained...
That these models/modules need to be available for training and also use it doesn't make sense to blackbox them to use only.
if I was going to create a end-to-end DL voice assistant the speech-enhancement would be finetuned to the wakeword by including that wakeword in the speech-enhancement dataset. That data is already available from the initial wakeword dataset. Then the wakeword is retrained with a dataset augmented by the speech-enhancement and also ASR so that its is a end-to-end DL voice assistant, than merely a collection of general purpose speech modules black-boxed as a supposed voice-assistant.
This is what consumer level voice-assistants have been doing for over a decade with ever better forms of speech-enhancement and they create tuned end-to-end DL architectures, whilst opensource provided just doesn't.
Beta Was this translation helpful? Give feedback.
All reactions