ml5-library
ml5-library copied to clipboard
ml5js
SoundClassifier warns about Mismatch in sampling rate
Running a sound classifier against a model on Teachable Machine gives me the following error message on a MacBook Air with a current version of Chrome:
Mismatch in sampling rate: Expected: 44100; Actual: 48000
I believe this was fixed in speech-commands in March 2021 (https://github.com/tensorflow/tfjs-models/commit/0b4801f6f5fa72e1859344c6c70c9236aaa85e06), which is in released versions 0.5.4 and above.
I hackishly fixed this in the latest release by applying the patch above manually. This version doesn't give me the error above (and works):
https://unpkg.com/@gohai/ml5@latest/dist/ml5.min.js
Hi @gohai thanks for reporting this, could you be so kind and open a PR with your hacky fix?! I think it might be easier for us to look at it that way. Gonna link @joeyklee as well.
Thanks
I just modified the dependency source. One could try to bump the version of @tensorflow-models/speech-commands from 0.3.9 to 0.5.4, but I know too little about this to say if this depends on the main TF version being used.
Indeed, when I try npm installed after bumping the version:
npm WARN @tensorflow-models/[email protected] requires a peer of @tensorflow/tfjs-core@^3.0.0 but none is installed. You must install peer dependencies yourself.
npm WARN @tensorflow-models/[email protected] requires a peer of @tensorflow/tfjs-data@^3.0.0 but none is installed. You must install peer dependencies yourself.
npm WARN @tensorflow-models/[email protected] requires a peer of @tensorflow/tfjs-layers@^3.0.0 but none is installed. You must install peer dependencies yourself.
I also not that much into this, but @joeyklee is currently very deeply involved in the core. Can you maybe explain a little bit more what you did to get this error and what you are trying to achieve. This way this issue can might be helpful for future readers. Thanks!
Of course: it looks like the audio context in SoundClassifier gets initialized with an unspecified sample rate, which ended up being 48 kHz on my MacBook Air running macOS and Chrome. It seems like the model assumes a 44.1 kHz sample rate - so this produces the warning Mismatch in sampling rate when running a sketch that uses SoundClassifier. (I wasn't able to verify that this impacts inference performance, but I assume it does.)
A later version of tensorflow-models/speech-commands modifies the audio context constructor to request the sample rate the model is using. Warning is gone.
Hi there! I will be away from the office for the next week, but I will try to look into this as well next week. TY!
Closing this issue since we'll be using an updated version of TF in the new repo!
