TensorRT
TensorRT copied to clipboard
NVIDIA
DLA for Digit Recognition With Dynamic Shapes
Description
No layer in sampleDynamicReshape does not run on DLA. Is there any easy way to run these layers on DLA?
Environment
JetPack 5.0.1 on AGX Orin and Jetpack 4.6 on Xavier AGX
Steps To Reproduce
Instructions here are followed: https://github.com/NVIDIA/TensorRT/tree/release/7.1/samples/opensource/sampleMNIST Engine logs are inserted below. Any layer can't run on DLA (I have several examples running on DLA for other networks).
Related Engine logs:
&&&& RUNNING TensorRT.sample_dynamic_reshape # /home/ismetdagli/temp_tensorrt/bin/sample_dynamic_reshape --datadir /home/ismetdagli/temp_tensorrt/data/mnist --fp16 --useDLACore=0
Input filename: /home/ismetdagli/temp_tensorrt/data/mnist/mnist.onnx ONNX IR version: 0.0.3 Opset version: 8 Producer name: CNTK Producer version: 2.5.1 Domain: ai.cntk Model version: 1
[W] [TRT] onnx2trt_utils.cpp:220: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32. [I] [TRT] [I] [TRT] --------------- Layers running on DLA: [I] [TRT] [I] [TRT] --------------- Layers running on GPU: [I] [TRT] Convolution28 + ReLU32, Pooling66, Convolution110 + ReLU114, Pooling160, Times212_reshape0, (Unnamed Layer* 0) [Constant] + Times212_reshape1, Times212, (Unnamed Layer* 16) [Constant], Plus214, (Unnamed Layer* 18) [Softmax], [I] [TRT] Detected 1 inputs and 1 output network tensors. [W] [TRT] DLA requests all profiles have same min, max, and opt value. All dla layers are falling back to GPU [I] [TRT] [I] [TRT] --------------- Layers running on DLA: [I] [TRT] [I] [TRT] --------------- Layers running on GPU: [I] [TRT] (Unnamed Layer* 0) [Resize], [I] [TRT] Detected 1 inputs and 1 output network tensors. [I] Profile dimensions in preprocessor engine: [I] Minimum = (1, 1, 1, 1) [I] Optimum = (1, 1, 28, 28) [I] Maximum = (1, 1, 56, 56) [I] Input:
DLA does not support dynamic dimensions. Thus, for wildcard dimensions, the min, max, and opt values of the profile must be equal.. see https://docs.nvidia.com/deeplearning/tensorrt/developer-guide/index.html#dla-supp-lay-rest
closing since no activity for more than 3 weeks, please reopen if you still have question, thanks!
