benchmark_model mobilenet with GPU wont run

[SOLEROM]

1. System information

• OS Platform and Distribution = Linux Ubuntu) on coral dev
• TensorFlow installation= built from source):
• TensorFlow library= if built from source):

2. code

benchmark_model from tflite build from source using -DTFLITE_ENABLE_GPU=ON in cmake cmd -DCL_DELEGATE_NO_GL in ARMCC_FLAGS

imx-gpu-viv installed on coral

clinfo show opencl1.2 and the device

but running the benchmark_model with use gpu wont work:

root@silly-finch:/# ./benchmark_model --use_gpu=true --graph=mobilenet_v1_1.0_224_float.tflite 
INFO: STARTING!
INFO: Log parameter values verbosely: [0]
INFO: Graph: [mobilenet_v1_1.0_224_float.tflite]
INFO: Use gpu: [1]
INFO: Loaded model mobilenet_v1_1.0_224_float.tflite
INFO: Created TensorFlow Lite delegate for GPU.
INFO: GPU delegate created.
ERROR: Failed to build program executable - Build program failure(97:0) : error : conversion between different vector types not allowed
(98:0) : error : conversion between different vector types not allowed
(99:0) : error : conversion between different vector types not allowed

will be glad for any tips... thanks;

[sushreebarsa]

@SOLEROM Could you check if you have built TFLite with -DTFLITE_ENABLE_GPU=ON during conversion? Building TFLite without GPU support will prevent GPU usage. Please ensure your GPU is properly configured and recognized by TensorFlow. You can check in the TensorFlow shell with tf.config.list_physical_devices('GPU'). Please let us know! Thank you!

[SOLEROM]

(1) our TFLite app benchmark_model was indeed build with -DTFLITE_ENABLE_GPU=ON ;

(2) the python tflite (from pip3 install tensorflow-io tensorflow) dont see any GPU;

tf.config.list_physical_devices() [PhysicalDevice(name='/physical_device:CPU:0', device_type='CPU')]

Checked with hello world opencl kernel example to make sure the hw is working.

Is there any configs to tflite to set the GPU? or any on the coral DEV maybe? Thank you!

[sushreebarsa]

@SOLEROM TFLite leverages "delegates" to offload computations to hardware accelerators like GPUs. You'll need to add the appropriate GPU delegate library to your project, depending on your platform (Android, iOS, etc.). Please refer to this doc for more details. Thank you!

[SOLEROM]

just to make sure: is compiling the app with -DTFLITE_ENABLE_GPU=ON in cmake cmd wont create the "deleage" for the GPU? running that app log that GPU delegate created and loaded :

root@silly-finch:/# ./benchmark_model --use_gpu=true --graph=mobilenet_v1_1.0_224_float.tflite 
...


INFO: Use gpu: [1]
INFO: Loaded model mobilenet_v1_1.0_224_float.tflite
INFO: Created TensorFlow Lite delegate for GPU.
INFO: GPU delegate created.                           <<<<========

....

[sushreebarsa]

@SOLEROM Compiling the app with -DTFLITE_ENABLE_GPU=ON in the cmake command won't create the delegate for the GPU. While enabling GPU support builds the necessary libraries for GPU execution, it doesn't automatically inject the delegate into your model. Thank you!

[github-actions[bot]]

This issue is stale because it has been open for 7 days with no activity. It will be closed if no further activity occurs. Thank you.

[github-actions[bot]]

This issue was closed because it has been inactive for 7 days since being marked as stale. Please reopen if you'd like to work on this further.