Using performance of 4 Google Coral USB Accellerator

[semaf]

Hey,

I was testing the performance of 2x Google Coral USB Accelerator on Raspberry Pi where a single Coral USB Accelerator recognized the macaw picture with 0.75781 score and both of the Coral USB Accelerator it at 0.99609. What other benefits could I get if I run 4 pieces of the Coral USB Accelerator? Faster results? Better scoring? Higher FPS?

Read a lot of using single Coral USB Accelerator but couldn't get anything about multiple of them. Thanks a lot!

Result of single USB Accelerator

pi@coral:~/coral/pycoral $ python3 examples/classify_image.py \
--model test_data/mobilenet_v2_1.0_224_inat_bird_quant_edgetpu.tflite \
--labels test_data/inat_bird_labels.txt \
--input test_data/parrot.jpg
----INFERENCE TIME----
Note: The first inference on Edge TPU is slow because it includes loading the model into Edge TPU memory.
18.4ms
5.2ms
5.3ms
5.2ms
5.2ms
-------RESULTS--------
Ara macao (Scarlet Macaw): 0.75781

Result of 2x USB Accelerator

pi@coral:~/coral/pycoral $ python3 examples/model_pipelining_classify_image.py  \
--model     test_data/pipeline/inception_v3_299_quant_segment_%d_of_2_edgetpu.tflite \   
--labels test_data/imagenet_labels.txt  \
--input test_data/parrot.jpg \
Using devices:  ['usb:0', 'usb:1']
Using models:  ['test_data/pipeline/inception_v3_299_quant_segment_0_of_2_edgetpu.tflite', 'test_data/pipeline/inception_v3_299_quant_segment_1_of_2_edgetpu.tflite']
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20220926 21:20:31.362005  4268 pipelined_model_runner.cc:172] Thread: 547667042752 receives empty request
I20220926 21:20:31.362146  4268 pipelined_model_runner.cc:245] Thread: 547667042752 is shutting down the pipeline...
I20220926 21:20:31.610878  4268 pipelined_model_runner.cc:255] Thread: 547667042752 Pipeline is off.
I20220926 21:20:31.611143  4269 pipelined_model_runner.cc:207] Queue is empty and `StopWaiters()` is called.
-------RESULTS--------
macaw: 0.99609
Average inference time (over 5 iterations): 51.0ms
I20220926 21:20:31.611727  4249 pipelined_model_runner.cc:172] Thread: 548061556800 receives empty request
E20220926 21:20:31.611774  4249 pipelined_model_runner.cc:240] Thread: 548061556800 Pipeline was turned off before.
Exception ignored in: <function PipelinedModelRunner.__del__ at 0x7f98bd69d0>
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/pycoral/pipeline/pipelined_model_runner.py", line 83, in __del__
    self.push({})
  File "/usr/lib/python3/dist-packages/pycoral/pipeline/pipelined_model_runner.py", line 152, in push
    self._runner.Push(input_tensors)
RuntimeError: Pipeline was turned off before.
E20220926 21:20:31.612711  4249 pipelined_model_runner.cc:240] Thread: 548061556800 Pipeline was turned off before.
E20220926 21:20:31.612746  4249 pipelined_model_runner.cc:147] Failed to shutdown status: INTERNAL: Pipeline was turned off before.

[hjonnala]

Hello @semaf running on multiple TPU module is useful when your input model is more than 8MB. If so, please try to compiling the model with -n flags (num_segments=model_size/8). The objective of this to make the inference faster for larger models (>>8MB). It won't change the accuracy of the model.

Also, please check the documentation at: https://coral.ai/docs/edgetpu/pipeline/