TensorRT icon indicating copy to clipboard operation
TensorRT copied to clipboard

🐛 [Bug] An assertion error when upgrading to enqueueV3 interface

Open zewenli98 opened this issue 1 year ago • 0 comments

Bug Description

https://github.com/pytorch/TensorRT/blob/main/py/torch_tensorrt/dynamo/runtime/_PythonTorchTensorRTModule.py#L63-L65

This should be:

assert (
    self.engine.num_io_tensors == len(self.input_names) + len(self.output_names)
)

because num_io_tensors is not inflated by the number of optimization profiles, for enqueueV3 interface. This is different from the enqueueV2 interface:

assert engine.num_bindings == num_io_tensors * engine.num_optimization_profiles

zewenli98 avatar May 03 '24 23:05 zewenli98