Tf 2.2.0rc3 fails to build TRT-Engine

[cbodenst]

Hi,

I have a quite large model (140MB weights) that, that was successfully converted to TRT with TF 2.1.0

But with 2.2.0 rc1,rc2,rc3 it always fails with following message Engine creation for TRTEngineOp_2 failed. The native segment will be used instead. Reason: Unimplemented: Transpose too large:10437228

I need a fix from the latest version (convert stateful lstm from keras to saved model), so going back is not an option.

What could be the reason that TF 2.2.0 is failing?

[tfeher]

Hi, there was recently a change that disabled TRT conversion of transpose ops on tensors that have more than 2.5 million elements. Normally this would cause the network to be converted into multiple TensorRT ops, but it should not fail. We will have a look this problem.

Until this is fixed, you can try to export the following environment variable, and see if this solves your problem: TF_DEBUG_TRT_ALLOW_INEFFICIENT_TRANSPOSE=1

[cbodenst]

@tfeher Thank you, setting the environment variable works indeed

[bharadwaj-vedula]

@cbodenst hey! how did you convert your model using TF 2.1.0 could u help me ?

[cbodenst]

@bharadwaj-vedula offtopic! but take a look to this guide: https://docs.nvidia.com/deeplearning/frameworks/tf-trt-user-guide/index.html

[Ekta246]

Hi,

I have a quite large model (140MB weights) that, that was successfully converted to TRT with TF 2.1.0

But with 2.2.0 rc1,rc2,rc3 it always fails with following message Engine creation for TRTEngineOp_2 failed. The native segment will be used instead. Reason: Unimplemented: Transpose too large:10437228

I need a fix from the latest version (convert stateful lstm from keras to saved model), so going back is not an option.

What could be the reason that TF 2.2.0 is failing?

@cbodenst On which GPU platform are you trying with TF versions 2.2.0rc1,rc2 and so on. Also, I didn't get success with TF 2.1.0 on Jetson AGX Xavier platform while using TF-TRT implementation.

[cbodenst]

Hi Ekta. was using Google Cloud Platform withn Tesla P100

[Ekta246]

Thanks for getting back real quick. Any luck with TensorRT on Jetson platforms?

Thanks

On Wed, Jul 8, 2020, 5:43 AM cbodenst [email protected] wrote:

Hi Ekta. was using Google Cloud Platform withn Tesla P100

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/tensorflow/tensorrt/issues/200#issuecomment-655412106, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMQBWSPY2LXMKBIWFEXRPJ3R2Q5UVANCNFSM4MT5GEDA .