TensorRT
TensorRT copied to clipboard
Published
20 hours ago •
NVIDIA
NVIDIA
Jetson Nano : Error TensorRT : Error Code 10: Internal Error (Could not find any implementation for node StatefulPartitionedCall/ArgMax.)
Hello,
After compiling tensorRT 8.4.2 from source, and try to convert my onnx model to TRT model on my jetson nano I got the following error :
Internal Error (Could not find any implementation for node StatefulPartitionedCall/ArgMax.)
Onnx model : https://drive.google.com/file/d/1i56egDoGe-xCTo2Q63yQWFiCdD-gaDt-/view?usp=sharing
NB : I try to use --opset 17 for creationg my onnx model but same problem... NB : I try also --workspace=512 but same problem...
The conversion work great under google colab using the same onnx model.
Can you please help me ?
!/home/jetson/TensorRT/build/trtexec --fp16 --workspace=4096 --onnx="model.onnx" --saveEngine="model_jetson.engine"
[02/24/2024-20:42:37] [W] --workspace flag has been deprecated by --memPoolSize flag.
[02/24/2024-20:42:37] [I] === Model Options ===
[02/24/2024-20:42:37] [I] Format: ONNX
[02/24/2024-20:42:37] [I] Model: model.onnx
[02/24/2024-20:42:37] [I] Output:
[02/24/2024-20:42:37] [I] === Build Options ===
[02/24/2024-20:42:37] [I] Max batch: explicit batch
[02/24/2024-20:42:37] [I] Memory Pools: workspace: 4096 MiB, dlaSRAM: default, dlaLocalDRAM: default, dlaGlobalDRAM: default
[02/24/2024-20:42:37] [I] minTiming: 1
[02/24/2024-20:42:37] [I] avgTiming: 8
[02/24/2024-20:42:37] [I] Precision: FP32+FP16
[02/24/2024-20:42:37] [I] LayerPrecisions:
[02/24/2024-20:42:37] [I] Calibration:
[02/24/2024-20:42:37] [I] Refit: Disabled
[02/24/2024-20:42:37] [I] Sparsity: Disabled
[02/24/2024-20:42:37] [I] Safe mode: Disabled
[02/24/2024-20:42:37] [I] DirectIO mode: Disabled
[02/24/2024-20:42:37] [I] Restricted mode: Disabled
[02/24/2024-20:42:37] [I] Build only: Disabled
[02/24/2024-20:42:37] [I] Save engine: model_jetson.engine
[02/24/2024-20:42:37] [I] Load engine:
[02/24/2024-20:42:37] [I] Profiling verbosity: 0
[02/24/2024-20:42:37] [I] Tactic sources: Using default tactic sources
[02/24/2024-20:42:37] [I] timingCacheMode: local
[02/24/2024-20:42:37] [I] timingCacheFile:
[02/24/2024-20:42:37] [I] Input(s)s format: fp32:CHW
[02/24/2024-20:42:37] [I] Output(s)s format: fp32:CHW
[02/24/2024-20:42:37] [I] Input build shapes: model
[02/24/2024-20:42:37] [I] Input calibration shapes: model
[02/24/2024-20:42:37] [I] === System Options ===
[02/24/2024-20:42:37] [I] Device: 0
[02/24/2024-20:42:37] [I] DLACore:
[02/24/2024-20:42:37] [I] Plugins:
[02/24/2024-20:42:37] [I] === Inference Options ===
[02/24/2024-20:42:37] [I] Batch: Explicit
[02/24/2024-20:42:37] [I] Input inference shapes: model
[02/24/2024-20:42:37] [I] Iterations: 10
[02/24/2024-20:42:37] [I] Duration: 3s (+ 200ms warm up)
[02/24/2024-20:42:37] [I] Sleep time: 0ms
[02/24/2024-20:42:37] [I] Idle time: 0ms
[02/24/2024-20:42:37] [I] Streams: 1
[02/24/2024-20:42:37] [I] ExposeDMA: Disabled
[02/24/2024-20:42:37] [I] Data transfers: Enabled
[02/24/2024-20:42:37] [I] Spin-wait: Disabled
[02/24/2024-20:42:37] [I] Multithreading: Disabled
[02/24/2024-20:42:37] [I] CUDA Graph: Disabled
[02/24/2024-20:42:37] [I] Separate profiling: Disabled
[02/24/2024-20:42:37] [I] Time Deserialize: Disabled
[02/24/2024-20:42:37] [I] Time Refit: Disabled
[02/24/2024-20:42:37] [I] Inputs:
[02/24/2024-20:42:37] [I] === Reporting Options ===
[02/24/2024-20:42:37] [I] Verbose: Disabled
[02/24/2024-20:42:37] [I] Averages: 10 inferences
[02/24/2024-20:42:37] [I] Percentile: 99
[02/24/2024-20:42:37] [I] Dump refittable layers:Disabled
[02/24/2024-20:42:37] [I] Dump output: Disabled
[02/24/2024-20:42:37] [I] Profile: Disabled
[02/24/2024-20:42:37] [I] Export timing to JSON file:
[02/24/2024-20:42:37] [I] Export output to JSON file:
[02/24/2024-20:42:37] [I] Export profile to JSON file:
[02/24/2024-20:42:37] [I]
[02/24/2024-20:42:37] [I] === Device Information ===
[02/24/2024-20:42:37] [I] Selected Device: NVIDIA Tegra X1
[02/24/2024-20:42:37] [I] Compute Capability: 5.3
[02/24/2024-20:42:37] [I] SMs: 1
[02/24/2024-20:42:37] [I] Compute Clock Rate: 0.9216 GHz
[02/24/2024-20:42:37] [I] Device Global Memory: 3963 MiB
[02/24/2024-20:42:37] [I] Shared Memory per SM: 64 KiB
[02/24/2024-20:42:37] [I] Memory Bus Width: 64 bits (ECC disabled)
[02/24/2024-20:42:37] [I] Memory Clock Rate: 0.01275 GHz
[02/24/2024-20:42:37] [I]
[02/24/2024-20:42:37] [I] TensorRT version: 8.4.2
[02/24/2024-20:42:39] [I] [TRT] [MemUsageChange] Init CUDA: CPU +229, GPU +0, now: CPU 248, GPU 2724 (MiB)
[02/24/2024-20:42:40] [I] [TRT] [MemUsageSnapshot] Begin constructing builder kernel library: CPU 248 MiB, GPU 2724 MiB
[02/24/2024-20:42:40] [I] [TRT] [MemUsageSnapshot] End constructing builder kernel library: CPU 278 MiB, GPU 2754 MiB
[02/24/2024-20:42:40] [I] Start parsing network model
[02/24/2024-20:42:40] [I] [TRT] ----------------------------------------------------------------
[02/24/2024-20:42:40] [I] [TRT] Input filename: model.onnx
[02/24/2024-20:42:40] [I] [TRT] ONNX IR version: 0.0.8
[02/24/2024-20:42:40] [I] [TRT] Opset version: 15
[02/24/2024-20:42:40] [I] [TRT] Producer name: tf2onnx
[02/24/2024-20:42:40] [I] [TRT] Producer version: 1.16.1 9538da
[02/24/2024-20:42:40] [I] [TRT] Domain:
[02/24/2024-20:42:40] [I] [TRT] Model version: 0
[02/24/2024-20:42:40] [I] [TRT] Doc string:
[02/24/2024-20:42:40] [I] [TRT] ----------------------------------------------------------------
[02/24/2024-20:42:40] [W] [TRT] parsers/onnx/onnx2trt_utils.cpp:369: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[02/24/2024-20:42:40] [W] [TRT] Tensor DataType is determined at build time for tensors not marked as input or output.
[02/24/2024-20:42:40] [W] [TRT] Tensor DataType is determined at build time for tensors not marked as input or output.
[02/24/2024-20:42:40] [I] Finish parsing network model
[02/24/2024-20:42:40] [I] [TRT] ---------- Layers Running on DLA ----------
[02/24/2024-20:42:40] [I] [TRT] ---------- Layers Running on GPU ----------
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/Cast
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/Reshape:0
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/truediv:0
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] PWN(StatefulPartitionedCall/sub_1/y:0 + (Unnamed Layer* 7) [Shuffle], PWN(PWN(StatefulPartitionedCall/sub, StatefulPartitionedCall/truediv_1), StatefulPartitionedCall/sub_1))
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/Conv1/Conv2D__15
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/Conv1/Conv2D + PWN(StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/Conv1_relu/Relu6)
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/expanded_conv_depthwise/depthwise + PWN(StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/expanded_conv_depthwise_relu/Relu6)
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/expanded_conv_project/Conv2D
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_1_expand/Conv2D + PWN(StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_1_expand_relu/Relu6)
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_1_depthwise/depthwise + PWN(StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_1_depthwise_relu/Relu6)
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_1_project/Conv2D
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_2_expand/Conv2D + PWN(StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_2_expand_relu/Relu6)
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_2_depthwise/depthwise + PWN(StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_2_depthwise_relu/Relu6)
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_2_project/Conv2D + StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_2_add/add
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_3_expand/Conv2D + PWN(StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_3_expand_relu/Relu6)
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_3_depthwise/depthwise + PWN(StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_3_depthwise_relu/Relu6)
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_3_project/Conv2D
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_4_expand/Conv2D + PWN(StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_4_expand_relu/Relu6)
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_4_depthwise/depthwise + PWN(StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_4_depthwise_relu/Relu6)
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_4_project/Conv2D + StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_4_add/add
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_5_expand/Conv2D + PWN(StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_5_expand_relu/Relu6)
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_5_depthwise/depthwise + PWN(StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_5_depthwise_relu/Relu6)
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_5_project/Conv2D + StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_5_add/add
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_6_expand/Conv2D + PWN(StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_6_expand_relu/Relu6)
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_6_depthwise/depthwise + PWN(StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_6_depthwise_relu/Relu6)
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_6_project/Conv2D
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_7_expand/Conv2D + PWN(StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_7_expand_relu/Relu6)
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_7_depthwise/depthwise + PWN(StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_7_depthwise_relu/Relu6)
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_7_project/Conv2D + StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_7_add/add
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_8_expand/Conv2D + PWN(StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_8_expand_relu/Relu6)
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_8_depthwise/depthwise + PWN(StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_8_depthwise_relu/Relu6)
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_8_project/Conv2D + StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_8_add/add
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_9_expand/Conv2D + PWN(StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_9_expand_relu/Relu6)
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_9_depthwise/depthwise + PWN(StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_9_depthwise_relu/Relu6)
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_9_project/Conv2D + StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_9_add/add
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_10_expand/Conv2D + PWN(StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_10_expand_relu/Relu6)
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_10_depthwise/depthwise + PWN(StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_10_depthwise_relu/Relu6)
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_10_project/Conv2D
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_11_expand/Conv2D + PWN(StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_11_expand_relu/Relu6)
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_11_depthwise/depthwise + PWN(StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_11_depthwise_relu/Relu6)
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_11_project/Conv2D + StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_11_add/add
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_12_expand/Conv2D + PWN(StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_12_expand_relu/Relu6)
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_12_depthwise/depthwise + PWN(StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_12_depthwise_relu/Relu6)
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_12_project/Conv2D + StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_12_add/add
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_13_expand/Conv2D + PWN(StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_13_expand_relu/Relu6)
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_13_depthwise/depthwise + PWN(StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_13_depthwise_relu/Relu6)
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_13_project/Conv2D
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_14_expand/Conv2D + PWN(StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_14_expand_relu/Relu6)
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_14_depthwise/depthwise + PWN(StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_14_depthwise_relu/Relu6)
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_14_project/Conv2D + StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_14_add/add
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_15_expand/Conv2D + PWN(StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_15_expand_relu/Relu6)
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_15_depthwise/depthwise + PWN(StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_15_depthwise_relu/Relu6)
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_15_project/Conv2D + StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_15_add/add
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_16_expand/Conv2D + PWN(StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_16_expand_relu/Relu6)
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_16_depthwise/depthwise + PWN(StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_16_depthwise_relu/Relu6)
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/block_16_project/Conv2D
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/Conv_1/Conv2D + PWN(StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/model/out_relu/Relu6)
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/conv2d/BiasAdd
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] Resize__349
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/conv2d_1/BiasAdd + StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/tf.__operators__.add/AddV2
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/separable_conv2d/separable_conv2d/depthwise
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/separable_conv2d/BiasAdd + StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/batch_normalization/FusedBatchNormV3 + StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/re_lu/Relu
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] Resize__364
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/conv2d_2/BiasAdd + StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/tf.__operators__.add_1/AddV2
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/separable_conv2d_1/separable_conv2d/depthwise
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/separable_conv2d_1/BiasAdd + StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/batch_normalization_1/FusedBatchNormV3 + StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/re_lu_1/Relu
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] Resize__379
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/conv2d_3/BiasAdd + StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/tf.__operators__.add_2/AddV2
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/separable_conv2d_2/separable_conv2d/depthwise
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/separable_conv2d_2/BiasAdd + StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/batch_normalization_2/FusedBatchNormV3 + StatefulPartitionedCall/center_net_mobile_net_v2fpn_feature_extractor/model_1/re_lu_2/Relu
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/center_0/separable_conv2d_3/separable_conv2d/depthwise
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/center_0/separable_conv2d_3/BiasAdd + StatefulPartitionedCall/center_0/re_lu_3/Relu
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/kpt_heatmap_0/separable_conv2d_4/separable_conv2d/depthwise
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/kpt_heatmap_0/separable_conv2d_4/BiasAdd + StatefulPartitionedCall/kpt_heatmap_0/re_lu_4/Relu
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/kpt_regress_0/separable_conv2d_5/separable_conv2d/depthwise
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/kpt_regress_0/separable_conv2d_5/BiasAdd + StatefulPartitionedCall/kpt_regress_0/re_lu_5/Relu
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/kpt_regress_0/conv2d_6/BiasAdd
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/kpt_heatmap_0/conv2d_5/BiasAdd
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/center_0/conv2d_4/BiasAdd
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] PWN(StatefulPartitionedCall/Sigmoid_1)
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/center_0/conv2d_4/BiasAdd__426
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] ConstantFolding/StatefulPartitionedCall/truediv_2_recip:0 + (Unnamed Layer* 152) [Shuffle] + StatefulPartitionedCall/truediv_2
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/Reshape_2
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/ArgMax
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] {ForeignNode[StatefulPartitionedCall/kpt_regress_0/conv2d_6/BiasAdd__399...StatefulPartitionedCall/Reshape_7]}
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/kpt_offset_0/separable_conv2d_6/separable_conv2d/depthwise
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/kpt_offset_0/separable_conv2d_6/BiasAdd + StatefulPartitionedCall/kpt_offset_0/re_lu_6/Relu
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/kpt_offset_0/conv2d_7/BiasAdd
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] StatefulPartitionedCall/ArgMax_1
[02/24/2024-20:42:40] [I] [TRT] [GpuLayer] {ForeignNode[(Unnamed Layer* 203) [Shuffle]...StatefulPartitionedCall/concat_1]}
[02/24/2024-20:42:41] [I] [TRT] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +158, GPU +160, now: CPU 458, GPU 2936 (MiB)
[02/24/2024-20:42:43] [I] [TRT] [MemUsageChange] Init cuDNN: CPU +241, GPU +239, now: CPU 699, GPU 3175 (MiB)
[02/24/2024-20:42:43] [I] [TRT] Local timing cache in use. Profiling results in this builder pass will not be stored.
[02/24/2024-20:43:57] [I] [TRT] Some tactics do not have sufficient workspace memory to run. Increasing workspace size may increase performance, please check verbose output.
[02/24/2024-20:46:59] [E] Error[10]: [optimizer.cpp::computeCosts::2011] Error Code 10: Internal Error (Could not find any implementation for node StatefulPartitionedCall/ArgMax.)
[02/24/2024-20:46:59] [E] Error[2]: [builder.cpp::buildSerializedNetwork::609] Error Code 2: Internal Error (Assertion enginePtr != nullptr failed. )
[02/24/2024-20:46:59] [E] Engine could not be created from network
[02/24/2024-20:46:59] [E] Building engine failed
[02/24/2024-20:46:59] [E] Failed to create engine from model or file.
[02/24/2024-20:46:59] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec [TensorRT v8402] # /home/jetson/TensorRT/build/trtexec --fp16 --workspace=4096 --onnx=model.onnx --saveEngine=model_jetson.engine
type or paste code here
I just realized that under colab it is version 8.6 while under jetson it is 8.4... I'll try to recompile 8.6 under Jetson to see if that solves the problem...
I am unable to compile trt8.5 on my jetson.
Please can someone explain me if the error is due to the fact that Argmax is not supported ?? I thought it was the case no ??
Did someone have an idea to fix this issue ? Do I need for example write my own pluggin to implement argmax and use surgeon to replace the Argmax by my own layer ?
Can someone give me some advice so I can do my own research ?
Thanks you.
How about just flash the JP to latest 6.x version, it should have 8.6 shipped. AFAIK we don't support install CUDA-X SDKs manually in Jetson. You need to upgrade the JP version.
Jetpack max version is 4.6.3 on Jetson nano I cannot use JP6 even using sdk ...
Do you think my idea to create my own plugin for argmax should work ? Is it really argmax which is not supported ? Thanks !
You can check our release note(or onnx-tensorrt operator and checkout the 8.4 branch), I think it should be supported in 8.4.
Yes you can implement it as a plugin if you can upgrade the JP.