Did anyone have successfully converted to onnx or tensorrt?

[angryhen]

[sainttelant]

Not yet, even converted it to .pt file deployed in the c++ program, any ideas?

[wxthss82]

Haven't converted to TensorRT.

[romil611]

I was able to convert to onnx.

• While creating the model we had to pass onnx_export=True.
• A couple of places that didn't shift swish from memoryEfficientSwish to Swish which I also linked with the onnx_export to change it.
• Changed #h, w = x.shape[-2:] to h, w = x.cpu().detach().numpy().shape[-2:].
• Opset >= 11

After doing this I get the onnx file. But, when converting to tensorRt, upSample is causing issues. If anyone can help with that then do tell me.

[dav-ell]

Upsample causing issues for me too, documenting here: https://github.com/NVIDIA/Torch-TensorRT/issues/961

[kongyanye]

I've successfully convert it to TensorRT version for inference. Kindly check it here: https://github.com/kongyanye/EfficientDet-TensorRT

[romil611]

Also checkout https://github.com/NVIDIA/TensorRT/tree/main/samples/python/efficientdet

[dav-ell]

I've successfully convert it to TensorRT version for inference. Kindly check it here: https://github.com/kongyanye/EfficientDet-TensorRT

Amazing work! Thanks very much for sharing. I noticed, though, that you didn't include fold_constants=True in your export to ONNX -- I had to include that or else my conversion to TensorRT would fail. I also had to be running a version of TensorRT newer than 7.1.3. I'd also be curious to see what kind of performance boost you're getting and if you've tried it with Triton? Will leave an issue on your repo.

Also checkout https://github.com/NVIDIA/TensorRT/tree/main/samples/python/efficientdet

Yes, I've been aware of this one, but it's based on the TensorFlow version. It's not so important that EfficientDet itself is exported, but that a PyTorch version that I can make changes to is exported.

---

In a related note, does anyone know if TensorRT supports efficient separable convolutions? This model has tons of them, and it slightly slows down performance without a special optimization like they have in TensorFlow.