Deployment on TX2

[ahangchen]

Hi Lingtong, the performance of FastFlowNet is surprising. I want to deploy it on TX2. But I find there's a custom layer (correlation layer), thus the model can not be convert to onnx directly. I also try torch2tensorRT, but fail on this layer. How do you deploy it on TX2? Can you share the code about model conversion and test code on TX2 using TensorRT？

[ltkong218]

Thanks for the interest of my work. Currently, I can not share the conversion and deployment codes on NVIDIA TX2. Some guidance: Since the correlation is custom layer which is not supported in original TensorRT, I add it as plugins and convert the PyTorch model to TensorRT version directly without using any conversion tools.

[ahangchen]

Can you provide the toolkit names about converting pytorch model to trt? I guess you convert torch to onnx with pytorch basic api and load onnx model in tensorRT?

[ltkong218]

I do not use onnx as an intermediate step but define a TRT model directly.

[jucic]

I do not use onnx as an intermediate step but define a TRT model directly.

How to define a TRT model directly? by torch2tensorRT?

[anenbergb]

@ahangchen @jucic did either of you manage to export FastFlowNet to TensorRT? Also, which version of TensorRT did you use (TensorRT 6 or 7?)?