kahst
Is it possible to use the Google coral with TFLite for BirdNET?
So I noticed #6 and the following commit would allow for specifying TFLite import. Would this also then open up for using the Google Coral to do the processing, or is the workload so low that this isn't benefitial? I already have a few of those in my cluster, and I thought it would be neat to feed audio from my outdoor cameras (RTSP) into BirdNET since I live in the middel of a bird migration "pit stop". But using as little resources as possible is always nice.
If you're unfamiliar with the Coral another great project is using it to detect objects in images with TFLite models: https://github.com/blakeblackshear/frigate/blob/master/frigate/edgetpu.py
That's a good question :) I never actually tried to run it on Coral board (I have one, but just never tried). Yet, I believe the model needs to be fully quantized in order to run on a Coral board which uses a TPU if I'm correct. Can you confirm that? If that's so, I guess it won't work because our INT8 version is only partially quantized through quantize aware training.
Ah, it seems you are right:
TensorFlow Lite models can be made even smaller and more efficient through quantization, which converts 32-bit parameter data into 8-bit representations (which is required by the Edge TPU). ref
From that article though it sounds as if one can convert a TF model to a TFLite model compatible with the EdgeTPU but I don't have enough ML knowledge to understand the implications or practical appraoch. But I'm guessing that there is a reason why your INT8 version is only partially quantized through quantize aware training, and so it won't be a simple "convert & go" in any case.
Yeah, that's what I think, it won't be straight forward. The reason why our models aren't fully quantized is that I couldn't make it work otherwise. Performance would just drop significantly when fully quantized, so I went with partial quantization. That still gives you a speed-up on Raspberry Pi or mobile phones compared to the FP32 version, it just won't run on TPU. Not sure if I can figure out how to do it, I might at some point :)