how to conver the .pth file to onnx？

[xinsuinizhuan]

💡 Your Question

I use the code to convert: from super_gradients.common.object_names import Models from super_gradients.training import models import torch

model = models.get(Models.YOLO_NAS_S, checkpoint_path='yolo_nas_s_coco.pth', num_classes=80)

model.eval() model.prep_model_for_conversion(input_size=[1, 3, 640, 640])

dummy_input = torch.randn(1, 3, 640, 640)

torch.onnx.export(model, dummy_input, "model.onnx")

but when i use the model.onnx file to convert to trt, onnxParser->parseFromFile could not parase the model.onnx

Versions

I use the code to convert: from super_gradients.common.object_names import Models from super_gradients.training import models import torch

model = models.get(Models.YOLO_NAS_S, checkpoint_path='yolo_nas_s_coco.pth', num_classes=80)

model.eval() model.prep_model_for_conversion(input_size=[1, 3, 640, 640])

dummy_input = torch.randn(1, 3, 640, 640)

torch.onnx.export(model, dummy_input, "model.onnx")

but when i use the model.onnx file to convert to trt, onnxParser->parseFromFile could not parase the model.onnx

[shaydeci]

Hey @xinsuinizhuan , please see our recently added benchmarking YoloNAS tutorial.

Our recently added quantization aware fine tuning YoloNAS on custom dataset notebook covers this as well.

[xinsuinizhuan]

could provide a export.py file or command as the yolov5 and yolov8? I also use the super_gradients.training, but he onnx could not use.

[xinsuinizhuan]

It still not work，i export the onnx model, as the below code:

from super_gradients.common.object_names import Models from super_gradients.training import models import torch model = models.get(Models.YOLO_NAS_S, checkpoint_path='yolo_nas_s_coco.pth', num_classes=80) model.eval() model.prep_model_for_conversion(input_size=[1, 3, 640, 640]) dummy_input = torch.randn(1, 3, 640, 640) torch.onnx.export(model, dummy_input, "model.onnx")

and i get the model.onnx file， then i use the below code convert to trt, but it break:

xxxxx\bin>trtexec --fp16 --int8 --avgRuns=100 --onnx=model.onnx &&&& RUNNING TensorRT.trtexec [TensorRT v8204] # trtexec --fp16 --int8 --avgRuns=100 --onnx=model.onnx [06/07/2023-14:00:56] [I] === Model Options === [06/07/2023-14:00:56] [I] Format: ONNX [06/07/2023-14:00:56] [I] Model: model.onnx [06/07/2023-14:00:56] [I] Output: [06/07/2023-14:00:56] [I] === Build Options === [06/07/2023-14:00:56] [I] Max batch: explicit batch [06/07/2023-14:00:56] [I] Workspace: 16 MiB [06/07/2023-14:00:56] [I] minTiming: 1 [06/07/2023-14:00:56] [I] avgTiming: 8 [06/07/2023-14:00:56] [I] Precision: FP32+FP16+INT8 [06/07/2023-14:00:56] [I] Calibration: Dynamic [06/07/2023-14:00:56] [I] Refit: Disabled [06/07/2023-14:00:56] [I] Sparsity: Disabled [06/07/2023-14:00:56] [I] Safe mode: Disabled [06/07/2023-14:00:56] [I] DirectIO mode: Disabled [06/07/2023-14:00:56] [I] Restricted mode: Disabled [06/07/2023-14:00:56] [I] Save engine: [06/07/2023-14:00:56] [I] Load engine: [06/07/2023-14:00:56] [I] Profiling verbosity: 0 [06/07/2023-14:00:56] [I] Tactic sources: Using default tactic sources [06/07/2023-14:00:56] [I] timingCacheMode: local [06/07/2023-14:00:56] [I] timingCacheFile: [06/07/2023-14:00:56] [I] Input(s)s format: fp32:CHW [06/07/2023-14:00:56] [I] Output(s)s format: fp32:CHW [06/07/2023-14:00:56] [I] Input build shapes: model [06/07/2023-14:00:56] [I] Input calibration shapes: model [06/07/2023-14:00:56] [I] === System Options === [06/07/2023-14:00:56] [I] Device: 0 [06/07/2023-14:00:56] [I] DLACore: [06/07/2023-14:00:56] [I] Plugins: [06/07/2023-14:00:56] [I] === Inference Options === [06/07/2023-14:00:56] [I] Batch: Explicit [06/07/2023-14:00:56] [I] Input inference shapes: model [06/07/2023-14:00:56] [I] Iterations: 10 [06/07/2023-14:00:56] [I] Duration: 3s (+ 200ms warm up) [06/07/2023-14:00:56] [I] Sleep time: 0ms [06/07/2023-14:00:56] [I] Idle time: 0ms [06/07/2023-14:00:56] [I] Streams: 1 [06/07/2023-14:00:56] [I] ExposeDMA: Disabled [06/07/2023-14:00:56] [I] Data transfers: Enabled [06/07/2023-14:00:56] [I] Spin-wait: Disabled [06/07/2023-14:00:56] [I] Multithreading: Disabled [06/07/2023-14:00:56] [I] CUDA Graph: Disabled [06/07/2023-14:00:56] [I] Separate profiling: Disabled [06/07/2023-14:00:56] [I] Time Deserialize: Disabled [06/07/2023-14:00:56] [I] Time Refit: Disabled [06/07/2023-14:00:56] [I] Skip inference: Disabled [06/07/2023-14:00:56] [I] Inputs: [06/07/2023-14:00:56] [I] === Reporting Options === [06/07/2023-14:00:56] [I] Verbose: Disabled [06/07/2023-14:00:56] [I] Averages: 100 inferences [06/07/2023-14:00:56] [I] Percentile: 99 [06/07/2023-14:00:56] [I] Dump refittable layers:Disabled [06/07/2023-14:00:56] [I] Dump output: Disabled [06/07/2023-14:00:56] [I] Profile: Disabled [06/07/2023-14:00:56] [I] Export timing to JSON file: [06/07/2023-14:00:56] [I] Export output to JSON file: [06/07/2023-14:00:56] [I] Export profile to JSON file: [06/07/2023-14:00:56] [I] [06/07/2023-14:00:57] [I] === Device Information === [06/07/2023-14:00:57] [I] Selected Device: NVIDIA GeForce RTX 3060 Laptop GPU [06/07/2023-14:00:57] [I] Compute Capability: 8.6 [06/07/2023-14:00:57] [I] SMs: 30 [06/07/2023-14:00:57] [I] Compute Clock Rate: 1.702 GHz [06/07/2023-14:00:57] [I] Device Global Memory: 6143 MiB [06/07/2023-14:00:57] [I] Shared Memory per SM: 100 KiB [06/07/2023-14:00:57] [I] Memory Bus Width: 192 bits (ECC disabled) [06/07/2023-14:00:57] [I] Memory Clock Rate: 7.001 GHz [06/07/2023-14:00:57] [I] [06/07/2023-14:00:57] [I] TensorRT version: 8.2.4 [06/07/2023-14:00:58] [I] [TRT] [MemUsageChange] Init CUDA: CPU +653, GPU +0, now: CPU 19562, GPU 1210 (MiB) [06/07/2023-14:00:59] [I] [TRT] [MemUsageSnapshot] Begin constructing builder kernel library: CPU 19619 MiB, GPU 1210 MiB [06/07/2023-14:00:59] [I] [TRT] [MemUsageSnapshot] End constructing builder kernel library: CPU 19827 MiB, GPU 1254 MiB [06/07/2023-14:00:59] [I] Start parsing network model [06/07/2023-14:00:59] [I] [TRT] ---------------------------------------------------------------- [06/07/2023-14:00:59] [I] [TRT] Input filename: model.onnx [06/07/2023-14:00:59] [I] [TRT] ONNX IR version: 0.0.4 [06/07/2023-14:00:59] [I] [TRT] Opset version: 9 [06/07/2023-14:00:59] [I] [TRT] Producer name: pytorch [06/07/2023-14:00:59] [I] [TRT] Producer version: 1.11.0 [06/07/2023-14:00:59] [I] [TRT] Domain: [06/07/2023-14:00:59] [I] [TRT] Model version: 0 [06/07/2023-14:00:59] [I] [TRT] Doc string: [06/07/2023-14:00:59] [I] [TRT] ---------------------------------------------------------------- [06/07/2023-14:00:59] [W] [TRT] onnx2trt_utils.cpp:366: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32. [06/07/2023-14:00:59] [I] [TRT] No importer registered for op: NonZero. Attempting to import as plugin. [06/07/2023-14:00:59] [I] [TRT] Searching for plugin: NonZero, plugin_version: 1, plugin_namespace: [06/07/2023-14:00:59] [E] [TRT] ModelImporter.cpp:773: While parsing node number 274 [NonZero -> "onnx::Transpose_819"]: [06/07/2023-14:00:59] [E] [TRT] ModelImporter.cpp:774: --- Begin node --- [06/07/2023-14:00:59] [E] [TRT] ModelImporter.cpp:775: input: "onnx::NonZero_818" output: "onnx::Transpose_819" name: "NonZero_274" op_type: "NonZero"

[06/07/2023-14:00:59] [E] [TRT] ModelImporter.cpp:776: --- End node --- [06/07/2023-14:00:59] [E] [TRT] ModelImporter.cpp:779: ERROR: builtin_op_importers.cpp:4870 In function importFallbackPluginImporter: [8] Assertion failed: creator && "Plugin not found, are the plugin name, version, and namespace correct?" [06/07/2023-14:00:59] [E] Failed to parse onnx file [06/07/2023-14:00:59] [I] Finish parsing network model [06/07/2023-14:00:59] [E] Parsing model failed [06/07/2023-14:00:59] [E] Failed to create engine from model. [06/07/2023-14:00:59] [E] Engine set up failed

[xinsuinizhuan]

how to convert the onnx to tensorrt, correclty?

[lpkoh]

I am facing same issue

[xinsuinizhuan]

I am facing same issue

No one repley this issue. So could not use it.

[zoltan-ongithub]

Here you go. Hope it helps:

    model = models.get("yolo_nas_s",
                        num_classes=NUM_CLASSES,
                        checkpoint_path=INPUT_FILE)

    models.convert_to_onnx(model=model, input_shape=(3,640,640), out_path="yolo_nas_s.onnx")

[BloodAxe]

Relevant issue: https://github.com/Deci-AI/super-gradients/issues/1333

[BloodAxe]

Export to ONNX with NMS is planned to be released in 3.2.0

https://github.com/Deci-AI/super-gradients/blob/master/documentation/source/models_export.md