yolov8x to engine fault

[adrianosantospb]

Hi, everyone.

I hope you are very well. I'm facing this problem.

Env

• GPU, e.g. V100, RTX2080, TX2, Xavier NX, Nano, etc. I'm using Docker with this image: FROM nvcr.io/nvidia/tensorrt:22.11-py3

About this repo

• which branch/tag/commit are you using? Main
• which model? yolov8 segmentation

Your problem

• what is your command? ./yolov8_seg -s yolov8x_seg.wts yolov8x_seg.engine x
• what's your output?

[02/20/2024-16:57:05] [E] [TRT] 4: [graphShapeAnalyzer.cpp::analyzeShapes::1872] Error Code 4: Miscellaneous (IShuffleLayer (Unnamed Layer* 509) [Shuffle]: reshape changes volume. Reshaping [32,56,56] to [32,6400].)
[02/20/2024-16:57:05] [E] [TRT] 4: [network.cpp::validate::2956] Error Code 4: Internal Error (Could not compute dimensions for (Unnamed Layer* 0) [Convolution]_output, because the network is not valid.)
[02/20/2024-16:57:05] [E] [TRT] 2: [builder.cpp::buildSerializedNetwork::751] Error Code 2: Internal Error (Assertion engine != nullptr failed. )
Build engine successfully!
yolov8_seg: /home/app/yolov8_seg.cpp:69: void serialize_engine(std::string&, std::string&, std::string&, float&, float&, int&): Assertion `serialized_engine' failed.
Aborted (core dumped)

• what output do you expect?

yolov8x_seg.engine file.

In sequence, I'd like to know if could I do the same process on Jetson?

Thank you all.

[wang-xinyu]

@adrianosantospb Can you try yolov8n or yolov8s?

[adrianosantospb]

@adrianosantospb Can you try yolov8n or yolov8s? For sure.

After several code corrections (references and CMake file) (p.s: I will do a merge request after...), I conducted a conversion experiment with all YOLOV8 models (from n to x). The conversion and inference worked as expected. However, when I use my model, an error occurs in the process. The difference is that my model has only a single class, unlike the standard YOLO model that comes with 80 classes.

Before anything else, I made the modification in config.h (kNumClass = 1; and kInputH = 448; kInputW = 448;) before compiling the code. The error I encounter is:

./yolov8_seg -s yolov8x-seg.wts yolov8x-seg.engine x
Loading weights: yolov8x-seg.wts
[02/22/2024-19:23:08] [E] [TRT] 4: [graphShapeAnalyzer.cpp::analyzeShapes::1285] Error Code 4: Miscellaneous (IShuffleLayer (Unnamed Layer* 509) [Shuffle]: reshape changes volume. Reshaping [32,56,56] to [32,6400].)
[02/22/2024-19:23:08] [E] [TRT] 4: [graphShapeAnalyzer.cpp::analyzeShapes::1285] Error Code 4: Miscellaneous (IShuffleLayer (Unnamed Layer* 509) [Shuffle]: reshape changes volume. Reshaping [32,56,56] to [32,6400].)
[02/22/2024-19:23:08] [E] [TRT] 4: [graphShapeAnalyzer.cpp::analyzeShapes::1285] Error Code 4: Miscellaneous (IShuffleLayer (Unnamed Layer* 509) [Shuffle]: reshape changes volume. Reshaping [32,56,56] to [32,6400].)
[02/22/2024-19:23:08] [E] [TRT] 4: [graphShapeAnalyzer.cpp::analyzeShapes::1285] Error Code 4: Miscellaneous (IShuffleLayer (Unnamed Layer* 509) [Shuffle]: reshape changes volume. Reshaping [32,56,56] to [32,6400].)
[02/22/2024-19:23:08] [E] [TRT] 4: [graphShapeAnalyzer.cpp::analyzeShapes::1285] Error Code 4: Miscellaneous (IShuffleLayer (Unnamed Layer* 509) [Shuffle]: reshape changes volume. Reshaping [32,56,56] to [32,6400].)
[02/22/2024-19:23:08] [E] [TRT] 4: [graphShapeAnalyzer.cpp::analyzeShapes::1285] Error Code 4: Miscellaneous (IShuffleLayer (Unnamed Layer* 509) [Shuffle]: reshape changes volume. Reshaping [32,56,56] to [32,6400].)
[02/22/2024-19:23:08] [E] [TRT] 4: [graphShapeAnalyzer.cpp::analyzeShapes::1285] Error Code 4: Miscellaneous (IShuffleLayer (Unnamed Layer* 509) [Shuffle]: reshape changes volume. Reshaping [32,56,56] to [32,6400].)
[02/22/2024-19:23:08] [E] [TRT] 4: [graphShapeAnalyzer.cpp::analyzeShapes::1285] Error Code 4: Miscellaneous (IShuffleLayer (Unnamed Layer* 509) [Shuffle]: reshape changes volume. Reshaping [32,56,56] to [32,6400].)
[02/22/2024-19:23:08] [E] [TRT] 4: [graphShapeAnalyzer.cpp::analyzeShapes::1285] Error Code 4: Miscellaneous (IShuffleLayer (Unnamed Layer* 509) [Shuffle]: reshape changes volume. Reshaping [32,56,56] to [32,6400].)
[02/22/2024-19:23:08] [E] [TRT] 4: [graphShapeAnalyzer.cpp::analyzeShapes::1285] Error Code 4: Miscellaneous (IShuffleLayer (Unnamed Layer* 509) [Shuffle]: reshape changes volume. Reshaping [32,56,56] to [32,6400].)
Building engine, please wait for a while...
[02/22/2024-19:23:08] [E] [TRT] 4: [graphShapeAnalyzer.cpp::analyzeShapes::1285] Error Code 4: Miscellaneous (IShuffleLayer (Unnamed Layer* 509) [Shuffle]: reshape changes volume. Reshaping [32,56,56] to [32,6400].)
[02/22/2024-19:23:08] [E] [TRT] 4: [network.cpp::validate::2835] Error Code 4: Internal Error (Could not compute dimensions for (Unnamed Layer* 0) [Convolution]_output, because the network is not valid.)
[02/22/2024-19:23:08] [E] [TRT] 2: [builder.cpp::buildSerializedNetwork::609] Error Code 2: Internal Error (Assertion enginePtr != nullptr failed. )
Build engine successfully!
yolov8_seg: /home/yolo/yolov8_seg.cpp:69: void serialize_engine(std::__cxx11::string&, std::__cxx11::string&, std::__cxx11::string&, float&, float&, int&): Assertion `serialized_engine' failed.
Aborted (core dumped)

Any idea?

[wang-xinyu]

Does (kNumClass = 1; and kInputH = 640; kInputW = 640;) work?

[adrianosantospb]

Does (kNumClass = 1; and kInputH = 640; kInputW = 640;) work?

Yes. it works. But I confirmed that the model output is (448 448). Maybe there is some variable with fixed code 640. I'm trying to fix it as well. If you have some idea...

[adrianosantospb]

I noticed that the seg model' parameters are:

scales: # model compound scaling constants, i.e. 'model=yolov8n-seg.yaml' will call yolov8-seg.yaml with scale 'n'
  # [depth, width, max_channels]
  n: [0.33, 0.25, 1024]
  s: [0.33, 0.50, 1024]
  m: [0.67, 0.75, 768]
  l: [1.00, 1.00, 512]
  **x: [1.00, 1.25, 512]**

According to: https://github.com/ultralytics/ultralytics/blob/main/ultralytics/cfg/models/v8/yolov8-seg.yaml And we have into the yolov8_seg.cpp:

if (std::string(argv[1]) == "-s" && argc == 5) {
        wts = std::string(argv[2]);
        engine = std::string(argv[3]);
        sub_type = std::string(argv[4]);
        if (sub_type == "n") {
          gd = 0.33;
          gw = 0.25;
          max_channels = 1024;
        } else if (sub_type == "s") {
          gd = 0.33;
          gw = 0.50;
          max_channels = 1024;
        } else if (sub_type == "m") {
          gd = 0.67;
          gw = 0.75;
          max_channels = 576;
        } else if (sub_type == "l") {
          gd = 1.0;
          gw = 1.0;
          max_channels = 512;
        } else if (sub_type == "x") {
          gd = 1.0;
          gw = 1.25;
          **max_channels = 640;**
        } else{
          return false;
        }

The max_channels is used in:

serialize_engine(wts_name, engine_name, sub_type, gd, gw, max_channels);

If I change 640 to 512, I got:

root@0224f5009897:/home/yolo/build# ./yolov8_seg -s yolov8x-seg.wts yolov8x-seg.engine x
Loading weights: yolov8x-seg.wts
[02/23/2024-15:56:44] [E] [TRT] 3: (Unnamed Layer* 124) [Convolution]:kernel weights has count 1843200 but 1474560 was expected
[02/23/2024-15:56:44] [E] [TRT] 4: (Unnamed Layer* 124) [Convolution]: count of 1843200 weights in kernel, but kernel dimensions (3,3) with 320 input channels, 512 output channels and 1 groups were specified. Expected Weights count is 320 * 3*3 * 512 / 1 = 1474560

[wang-xinyu]

@Rex-LK Can you help take a look?

[adrianosantospb]

@Rex-LK Can you help take a look?

Any idea?

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[wang-xinyu]

@adrianosantospb Seems @lindsayshuo had already fixed the issue, can you try again?

[tianclll]

(kNumClass = 1; and kInputH = 640; kInputW = 640;) 有效吗？

有效

[adrianosantospb]

@adrianosantospb Seems @lindsayshuo had already fixed the issue, can you try again?

Yes. I will do it.