NVIDIA
[Issue still exists in TRT 10.4] Fail to build the engine if input tensor number > 2^31
Description
The release note of 10.0.1 mentioned that the issue “UNets with tensors containing >2^31 elements may fail during the engine building step” was fixed. However, my model (nnUNet) has 1x32x512x512x512 input and it still failed to build the engine in TRT 10.4.
Environment
TensorRT Version: 10.4
Relevant Files
Model link: I uploaded a dummy model for testing here.
Similar case https://github.com/NVIDIA/TensorRT/issues/3815 https://github.com/NVIDIA/TensorRT/issues/4004
@dzzhang96 can you provide your OS and the exact command/snippet you use to build the engine? I'll instance an internal bug at that point, thanks.
@moraxu Hi, thanks for the reply. I am using Ubuntu 22.04, NVIDIA-SMI: 525.147.05 Driver Version: 525.147.05 CUDA Version: 12.0, RTX A4500 with 16G RAM. Sorry I cannot share my codes here but you may use trtexec to reproduce the error. :) Thanks again
@dzzhang96 I was told that Conv doesn't support > INT32_MAX volume size (see https://docs.nvidia.com/deeplearning/tensorrt/operators/docs/Convolution.html#volume-limits). The "fix" in release note of 10.0.1 may refer to other operators/cases.
@moraxu Hi thanks for the useful information! I assume that TensorRT is using this Conv operator (nvinfer1::IConvolutionLayer Class) when running inference. Because I did not use it directly in my codes. In the release note, it is hard to be convinced that a unet structure model does not use a Conv operator.
Yes, correct.
In the release note, it is hard to be convinced that a unet structure model does not use a Conv operator.
Makes sense, we can improve the release note moving forward.
@moraxu Thanks! It would be very helpful if the input tensor number could be > 2^31, which means we can input the original 512x512x512 image for inference without losing any resolution.
Noted, although the 2^31 limitation is from an internal library that we depend on..
@moraxu This is crippling with any model if you use high resolution inputs. What is the alternative solution? Only to modify the architecture and downsample everything?
