openai whisper

[SomeCodingUser]

Hi,

could you use this library to run openai whisper with a tflite model? In the examples there are always labels provided, but for whisper there would not be any labels I think?

Thanks!

[Caldarie]

Hi,

I had a quick look and it seems openai whisper uses pytorch. For it to be compatible with this plugin:

1. You’ll need to convert it to tflite. I believe it can be done if you follow the instructions here
2. After converting the model, run the model through here to find input and output.
3. As for the labels.. you don’t need to have it. As long as the input can receive audio, and the output returns an array then it should work.
4. If input or output size does not match to this plugin, you may need to tinker the code to match your needs. Do let me know if you reach this stage

Also be aware that tflite does have a lot of limitations. For more information, read about operator compatibilities here. When converting from PyTorch to TensorFlow models, there may be some incompatibilities with your converted model on mobile devices

[nyadla-sys]

This is proof-of-concept project to create an Android app based on Whisper TFLite, which leverages the stock Android UI to show off its features. Whisper-TFLIte-Android-Example

Refer here for Whidper TFLite model https://github.com/usefulsensors/openai-whisper/blob/main/models/whisper.tflite

go thru README here https://github.com/usefulsensors/openai-whisper

[Caldarie]

@nyadla-sys Thanks for the share. :)

[nyadla-sys]

Feel free to download the openai/whisper-tiny tflite-based Android Whisper ASR APP from Google App Store.