willow
willow copied to clipboard
Implement stream/VAD timeout
I am still seeing cases where VAD/wake end never happens.
In testing I had a device stream continuously for 12 hours before I finally pulled the power...
It's very difficult to reproduce but:
-
It makes testing/development frustrating for me. I believe my issues are related to wifi environment but I have experienced this in multiple completely different environments - Wisconsin (two Unifi APs) and California (Qualcomm and Meditek AP). FWIR stintel has yet to observe this issue.
-
We don't (currently) have a use case where endless streaming makes sense anyway.
We should implement a timeout based on either seconds (10, maybe?), number of audio chunks, something to timeout streaming no matter what.
It seems that if I wait for a few seconds after the display indicates ready it's much less likely to occur.
I do sometimes press the cancel button because VAD/WAKE end don't trigger. This happens very often right after booting the device. This might be related to https://github.com/espressif/esp-sr/issues/47#issuecomment-1259365973:
Why we choose raw data for the first time is that the BSS algorithm need to take some time to converge.
I've also seen some variations. Say wake word and then don't say anything:
I (00:16:15.053) WILLOW: AUDIO_REC_WAKEUP_START
I (00:16:25.053) WILLOW: AUDIO_REC_WAKEUP_END
In this case "Say command..." stays on the display, and the backlight stays on, but we can say a new command and it will work.
Another variation I'm seeing when just saying the wake word and nothing after:
I (00:25:01.486) WILLOW: AUDIO_REC_WAKEUP_START
W (28278) AFE_SR: ERROR! rb_out slow!!!
W (28338) AFE_SR: ERROR! rb_out slow!!!
W (28398) AFE_SR: ERROR! rb_out slow!!!
...
The message keeps repeating and the device is unresponsive.
Another variation I'm seeing when just saying the wake word and nothing after:
I (00:25:01.486) WILLOW: AUDIO_REC_WAKEUP_START W (28278) AFE_SR: ERROR! rb_out slow!!! W (28338) AFE_SR: ERROR! rb_out slow!!! W (28398) AFE_SR: ERROR! rb_out slow!!! ...
The message keeps repeating and the device is unresponsive.
This one appears to have been caused by CONFIG_COMPILER_OPTIMIZATION_LEVEL_DEBUG=y in my sdkconfig.