Getting CPP TensorRT example to work/garbage output

[charlesrwest]

Hello!

I'm trying to get PyTorch-ONNX-TensorRT-CPP working on Manjaro Linux (cuda 11.0, opencv 4.4.0 with cuda support, tensorrt 7.2, pytorch 1.6). There have been a few problems and I'm not sure what to try next:

1. It compiles fine, but some of the used features are depreciated: [ 50%] Building CXX object CMakeFiles/trt_sample.dir/trt_sample.cpp.o /home/charlesrwest/cpp/libraries/learnopencv/PyTorch-ONNX-TensorRT-CPP/trt_sample.cpp: In function ‘void parseOnnxModel(const string&, TRTUniquePtr<nvinfer1::ICudaEngine>&, TRTUniquePtr<nvinfer1::IExecutionContext>&)’: /home/charlesrwest/cpp/libraries/learnopencv/PyTorch-ONNX-TensorRT-CPP/trt_sample.cpp:144:79: warning: ‘virtual nvinfer1::INetworkDefinition* nvinfer1::IBuilder::createNetwork()’ is deprecated [-Wdeprecated-declarations] 144 | TRTUniquePtr<nvinfer1::INetworkDefinition> network{builder->createNetwork()}; | ^ In file included from /home/charlesrwest/cpp/libraries/learnopencv/PyTorch-ONNX-TensorRT-CPP/trt_sample.cpp:3: /usr/include/NvInfer.h:6954:58: note: declared here 6954 | TRT_DEPRECATED virtual nvinfer1::INetworkDefinition* createNetwork() TRTNOEXCEPT = 0; | ^~~~~~~~~~~~~ [100%] Linking CXX executable trt_sample [100%] Built target trt_sample

2. Attempting to run it with the given onnx file fails

build]$ ./trt_sample ../resnet50.onnx ../turkish_coffee.jpg [libprotobuf ERROR /home/charlesrwest/cpp/libraries/tensorrt/src/build/third_party.protobuf/src/third_party.protobuf/src/google/protobuf/text_format.cc:309] Error parsing text-format onnx2trt_onnx.ModelProto: 1:9: Message type "onnx2trt_onnx.ModelProto" has no field named "version". Failed to parse ONNX model from file../resnet50.onnx Network must have at least one output Network validation failed. Segmentation fault (core dumped) 3. Updating the model succeeds:

python3 ./pytorch_model.py class: cup , confidence: 92.43077850341797 %, index: 968 class: espresso , confidence: 6.13804292678833 %, index: 967 class: coffee mug , confidence: 0.7285550832748413 %, index: 504 Model was successfully converted to ONNX format. It was saved to resnet50.onnx 4. But running on the updated model fails

`./trt_sample ../resnet50.onnx ../turkish_coffee.jpg

Input filename: ../resnet50.onnx ONNX IR version: 0.0.6 Opset version: 9 Producer name: pytorch Producer version: 1.6 Domain:
Model version: 0 Doc string:

ERROR: /home/charlesrwest/cpp/libraries/tensorrt/src/TensorRT/parsers/onnx/ModelImporter.cpp:500 In function importModel: [4] Assertion failed: !_importer_ctx.network()->hasImplicitBatchDimension() && "This version of the ONNX parser only supports TensorRT INetworkDefinitions with an explicit batch dimension. Please ensure the network was created using the EXPLICIT_BATCH NetworkDefinitionCreationFlag." ERROR: could not parse the model. Segmentation fault (core dumped) 5. This can be fixed by adding a flag when building the engine: TRTUniquePtrnvinfer1::INetworkDefinition network{builder->createNetwork()}; -> const auto explicitBatch = 1U << static_cast<uint32_t>(nvinfer1::NetworkDefinitionCreationFlag::kEXPLICIT_BATCH); TRTUniquePtrnvinfer1::INetworkDefinition network{builder->createNetworkV2(explicitBatch)}; ` 6. The teensorrt netwrk then runs, but gives a garbage output

` ./build/trt_sample ./resnet50.onnx ./turkish_coffee.jpg

Input filename: ./resnet50.onnx ONNX IR version: 0.0.6 Opset version: 9 Producer name: pytorch Producer version: 1.6 Domain:
Model version: 0 Doc string:

class: web site, website, internet site, site | confidence: 5.84856% | index: 916 class: cleaver, meat cleaver, chopper | confidence: 3.78223% | index: 499 class: nematode, nematode worm, roundworm | confidence: 3.32261% | index: 111 class: envelope | confidence: 3.00015% | index: 549 class: hatchet | confidence: 1.46123% | index: 596 class: nail | confidence: 1.37108% | index: 677 class: whistle | confidence: 1.23372% | index: 902 class: can opener, tin opener | confidence: 1.171% | index: 473 class: hook, claw | confidence: 1.13449% | index: 600 class: guillotine | confidence: 0.982188% | index: 583 class: theater curtain, theatre curtain | confidence: 0.93405% | index: 854 class: letter opener, paper knife, paperknife | confidence: 0.930356% | index: 623 class: panpipe, pandean pipe, syrinx | confidence: 0.855095% | index: 699 class: revolver, six-gun, six-shooter | confidence: 0.823099% | index: 763 class: safety pin | confidence: 0.822059% | index: 772 class: plunger, plumber's helper | confidence: 0.785783% | index: 731 class: scabbard | confidence: 0.769268% | index: 777 class: hair slide | confidence: 0.726561% | index: 584 class: bulletproof vest | confidence: 0.715805% | index: 465 class: table lamp | confidence: 0.715662% | index: 846 class: book jacket, dust cover, dust jacket, dust wrapper | confidence: 0.667637% | index: 921 class: assault rifle, assault gun | confidence: 0.663674% | index: 413 class: stopwatch, stop watch | confidence: 0.662865% | index: 826 class: abaya | confidence: 0.655316% | index: 399 class: binder, ring-binder | confidence: 0.63594% | index: 446 class: chain | confidence: 0.635571% | index: 488 class: dishwasher, dish washer, dishwashing machine | confidence: 0.618163% | index: 534 class: rule, ruler | confidence: 0.606973% | index: 769 class: fire screen, fireguard | confidence: 0.605063% | index: 556 class: lipstick, lip rouge | confidence: 0.594731% | index: 629 class: screw | confidence: 0.585789% | index: 783 class: muzzle | confidence: 0.576213% | index: 676 class: mousetrap | confidence: 0.57476% | index: 674 class: ocarina, sweet potato | confidence: 0.572207% | index: 684 class: magnetic compass | confidence: 0.535981% | index: 635 class: hammer | confidence: 0.524395% | index: 587 class: maraca | confidence: 0.512932% | index: 641 `

[zacharyfrederick]

Did you ever get this figured out? I am stuck with the same issue.

[ducdung98tb]

I have the same problem. Have you solved it yet?

[ducdung98tb]

auto input_width = dims.d[2]; auto input_height = dims.d[3]; auto channels = dims.d[1];

A change like above should solve the problem.

[seongkyun]

To short,

1. update the resnet50.onnx file by running 'python3 ./pytorch_model.py' in the terminal.
2. change the function named "parseOnnxModel" from TRTUniquePtr<nvinfer1::INetworkDefinition> network{builder->createNetwork()}; to const auto explicitBatch = 1U << static_cast<uint32_t>(nvinfer1::NetworkDefinitionCreationFlag::kEXPLICIT_BATCH); TRTUniquePtr<nvinfer1::INetworkDefinition> network{builder->createNetworkV2(explicitBatch)};
3. change the function named "preprocessImage" from auto input_width = dims.d[2]; auto input_height = dims.d[1]; auto channels = dims.d[0]; to auto input_width = dims.d[2]; auto input_height = dims.d[3]; auto channels = dims.d[1];
4. make it and run $ cmake -DOpenCV_DIR=/workspace/shared_data/opencv/opencv-4.5.1 -DTensorRT_DIR=/workspace/tensorrt .. $ make -j8 $ ./trt_sample ../resnet50.onnx ../turkish_coffee.jpg checked it on Ubuntu18.04 LTS, CUDA11.1, cuDNN8.1, OpenCV4.5.1 with cuda in nvidia-docker tensorrt20.10 image.