tensorrt No speed improvements after TF-TRT optimizing on a tensorflow BERT model

No speed improvements after TF-TRT optimizing on a tensorflow BERT model

Open SohaKhazaeli opened this issue 2 years ago • 0 comments

After optimizing the model with either FP32 or FP16 I don't get any speed improvements.

The optimization is done on tensorflow/tensorflow:2.10.0-gpu docker image. The model uses tensorflow-text and tf-models-official libraries

This is the log from optimization process:

Nov 07 '22 18:11 SohaKhazaeli