Object detection C++ FP16 inference

[dbrazey]

Hello,

I am working on object detection with tensorflow 1.14. I have the following training pipeline:

• training SSD mobilenet inception v2 300x300 on my custom dataset using the object detection API and check accuracy with tensorboard
• freeze the model with export_inference_graph.py located in the object detection module

I can use the generated file for inference with the tensorflow C++ API (1.14) on RTX 2080. Then, I would like to use TF-TRT for inference, via the C++ API.

• convert the frozen.pb to a compatible TFTRT model using TrtGraphConverter, as in the tutorial. I set precision_mode to FP32 or FP16
• use the generated file for inference in C++

=> frozen graph from training 100 FPS frozen graph TRT FP32 110 FPS frozen graph TRT FP16 110 FPS (no improvement)

The output of TRT convertion is

graph_size(MB)(native_tf): 15.0 graph_size(MB)(trt): 14.8 num_nodes(native_tf): 2515 num_nodes(tftrt_total): 1163 num_nodes(trt_only): 0 time(s) (trt_conversion): 0.7442

I note that num_nodes (trt_only) is zero.

Maybe I miss something or my pipeline is not correct ? Thanks for your help.

[ou525]

@dbrazey Where is the c ++ interface routine

[austingg]

@dbrazey the num_nodes(trt_only) = 0 says that your environment is something wrong. You may try tensorflow 1.15 with tensorrt included, or try nvidia tensorflow docker container.