maxbbraun
Google Coral support
The pre-trained Whisper models don't work out-of-the-box with the Google Coral Edge TPU. They would need to meet certain requirements so they can be converted to TensorFlow Lite, quantized to 8-bit, and compiled.
The usefulsensors/openai-whisper repo achieves part of that (e.g. whisper-int8.tflite), but does not produce compilable models yet (e.g. dynamic tensors need to be converted to static).
it may be worth to try whisper-tiny.en.tflite model on Google Coral Edge TPU
I think the challenge apart from the size is still that the converted models are not in a format that the compiler accepts, because they contain dynamic-size tensors.
If I am not missing anything,whisper-tiny.en.tflite model doesnot have dynamic-size tensors. Please open model using the netron app https://netron.app/
Oh, good to know that they differ that way. Thanks!
For reference, here's the compiler output for whisper-int8.tflite:
$ edgetpu_compiler whisper-int8.tflite
Edge TPU Compiler version 16.0.384591198
Started a compilation timeout timer of 180 seconds.
ERROR: Attempting to use a delegate that only supports static-sized tensors with a graph that has dynamic-sized tensors.
Compilation failed: Model failed in Tflite interpreter. Please ensure model can be loaded/run in Tflite interpreter.
Compilation child process completed within timeout period.
Compilation failed!
Unfortunately, whisper-tiny.en.tflite fails to compile for a different reason:
$ edgetpu_compiler whisper-tiny.en.tflite
Edge TPU Compiler version 16.0.384591198
ERROR: Op builtin_code out of range: 150. Are you using old TFLite binary with newer model?
ERROR: Registration failed.
Invalid model: whisper-tiny.en.tflite
Model could not be parsed
(I'm using this Colab to quickly try the compiler.)
maybe some of the op's are not supported by edgetpu_compiler.Maybe you can raise issue as part of edgetpu_compiler. as it is running on TFlite framework on rpi4/orange pi hardware and it may also work in coral hw.
@nyadla-sys hi I tested all your tflite models, they may work well in coral and i.mx8m plus hw under cpu delegate.But problem is under NPU/TPU/GPU delegate. would you have any comment? I suspect the problem lies in the input/output type. for example, when i work on i.mx8m plus NPU delegate, i got error "ERROR: begin_tensor, end_tensor and strides_tensor must be constant."
