TensorRT for EfficientNet model

[aditya1709]

I tried converting the EfficientNet-b4 model to TensorRT for optimized inference. It looks like there are some layers that don't have converters implemented and throws - AttributeError: 'Tensor' object has no attribute '_trt' Has anyone had luck with TensorRT or is there another way of achieving this?

[HoracceFeng]

Hi @aditya1709 , have you solved this problem? I am also trying to convert the model to trt4.1, but issue #47 claims that only trt > 5.1.5 works. Any updates?

[suman-19]

@aditya1709 Depthwise conv is better on CPU. Nvidia has not improved it.

[amlarraz]

Hey! I'm trying to convert the model to TensorRT (ver. 5.1.6.1) and surprisingly everything work in the conversion process but when I'm doing an inference using the .pth saved model gives me only zeros (no classes detected).

[ViswanathaReddyGajjala]

Is it possible to do the conversion process using a TensorRT 5.0.2.6 version? I have issues while converting the depth wise convolution layer. (Depth wise Conv layer is supported in the latest version.)

[korabelnikov]

Hey! I'm trying to convert the model to TensorRT (ver. 5.1.6.1) and surprisingly everythin

did you resolve it?

[amlarraz]

Nop...but with the update of the repo (the memory usage optimizat) I don't need TensorRT to work because the model fits in the Jetson's nano 4Gb of RAM

[pshwetank]

Hey! does this issue still persists with tensorRT conversion of Efficientnet. I tried converting Efficientnet-b7 to TensorRT(ver 7.0.0) and everythings works perfectly during conversion. Its only when I start inferencing, it shows a sharp drop in accuracy and inferences not matching the pytorch model inference. Are there anyone still facing the same issue.

[xuatpham]

@pshwetank I'm facing the same problem, did you solve it ?

[pshwetank]

@xuatpham Yeah...I feel that was mostly because of the preprocessing pipeline and less related to tensorRT model itself. Mostly, you will not see drop in performance if conversion was done without errors. Try to check the outputs from the torch model with trt model on same set of inputs. Chances are that they should match if the preprocessing pipeline is same.

[xuatpham]

@pshwetank thanks for your quick response. But it seems like I can't figure out what's going on. Can you share the script that you used to convert from eff to TRT and re-load it to inference? thanks a lot !!!

[pshwetank]

I took the usual path to convert Pytorch model --> Onnx model --> TRT model. Once that is done, you can use your pytorch preprocessing and convert the torch tensors into numpy arrays to feed them to TRT or onnx model. If you are using same preprocessing, you should not have any issues. I dont have specific scripts for the purpose but you can easily use onnxruntime to run onnx model and Torch-TRT to run TRT models.