Export to onnx

Open qiangxinglin opened this issue 1 year ago • 1 comments

I use the following script to export to ONNX

class ModelArgs:
    hidden_dims: List[int] = field(default_factory=lambda: [128]*3)
    n_downsample: int = 2
    mixed_precision: bool = True
    max_disp: int = 192
    n_gru_layers: int = 3
    corr_radius: int = 4
    corr_levels: int = 2
    valid_iters: int = 32

args = ModelArgs()
model_path = 'kitti15.pth'
model = nn.DataParallel(IGEVStereo(args), device_ids=[0])
model.load_state_dict(torch.load(model_path, weights_only=True), strict=True)
model = model.module
model = model.cuda()
model = model.eval()

onnx_file_path = "model.onnx"
input_tensor = torch.randn(1, 3, 256, 256)
scaled_tensor = (input_tensor - input_tensor.min()) / (input_tensor.max() - input_tensor.min()) * 255
scaled_tensor = scaled_tensor.to("cuda")

torch.onnx.export(
    model,
    (scaled_tensor, scaled_tensor), 
    onnx_file_path, 
    export_params=True,
    input_names=['input1', 'input2'],
    output_names=['output'],
    dynamic_axes={ 
        'input1': {0: 'B', 2: 'H', 3: 'W'},  
        'input2': {0: 'B', 2: 'H', 3: 'W'}, 
        'output': {0: 'B', 2: 'H', 3: 'W'},  
    }
)

It will give some warning:

/Selective-Stereo/Selective-IGEV/core/submodule.py:138: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if x.shape != rem.shape:
/Selective-Stereo/Selective-IGEV/core/submodule.py:153: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert C % num_groups == 0
/Selective-Stereo/Selective-IGEV/core/submodule.py:156: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert cost.shape == (B, num_groups, H, W)
/Selective-Stereo/Selective-IGEV/core/utils/utils.py:68: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert torch.unique(ygrid).numel() == 1 and H == 1 # This is a stereo problem
/Selective-Stereo/Selective-IGEV/core/igev_stereo.py:214: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if not self.training and itr < iters-1:
/Selective-Stereo/Selective-IGEV/core/submodule.py:67: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if x.shape != rem.shape:
/miniconda3/envs/pytorch/lib/python3.11/site-packages/torch/onnx/symbolic_helper.py:1531: UserWarning: ONNX export mode is set to TrainingMode.EVAL, but operator 'instance_norm' is set to train=True. Exporting with train=True.
  warnings.warn(

Then, I use the onnx model to do the inference will raise error when using CUDA Provider:

import onnxruntime as ort

providers = [
            'CUDAExecutionProvider',
        ]

sess = ort.InferenceSession("model.onnx", providers=providers)
result = sess.run(None, {'input1': image1.cpu().numpy(), 'input2': image2.cpu().numpy()})

2024-10-12 13:42:38.396724999 [W:onnxruntime:Default, scatter_nd.h:51 ScatterNDWithAtomicReduction] ScatterND with reduction=='none' only guarantees to be correct if indices are not duplicated.
2024-10-12 13:42:38.396776966 [W:onnxruntime:Default, scatter_nd.h:51 ScatterNDWithAtomicReduction] ScatterND with reduction=='none' only guarantees to be correct if indices are not duplicated.
2024-10-12 13:42:38.396829258 [W:onnxruntime:Default, scatter_nd.h:51 ScatterNDWithAtomicReduction] ScatterND with reduction=='none' only guarantees to be correct if indices are not duplicated.
2024-10-12 13:42:38.396883090 [W:onnxruntime:Default, scatter_nd.h:51 ScatterNDWithAtomicReduction] ScatterND with reduction=='none' only guarantees to be correct if indices are not duplicated.
2024-10-12 13:42:38.396940692 [W:onnxruntime:Default, scatter_nd.h:51 ScatterNDWithAtomicReduction] ScatterND with reduction=='none' only guarantees to be correct if indices are not duplicated.
2024-10-12 13:42:38.396997762 [W:onnxruntime:Default, scatter_nd.h:51 ScatterNDWithAtomicReduction] ScatterND with reduction=='none' only guarantees to be correct if indices are not duplicated.
2024-10-12 13:42:38.397050517 [W:onnxruntime:Default, scatter_nd.h:51 ScatterNDWithAtomicReduction] ScatterND with reduction=='none' only guarantees to be correct if indices are not duplicated.
2024-10-12 13:42:38.397104544 [W:onnxruntime:Default, scatter_nd.h:51 ScatterNDWithAtomicReduction] ScatterND with reduction=='none' only guarantees to be correct if indices are not duplicated.
2024-10-12 13:42:38.397157043 [W:onnxruntime:Default, scatter_nd.h:51 ScatterNDWithAtomicReduction] ScatterND with reduction=='none' only guarantees to be correct if indices are not duplicated.
2024-10-12 13:42:38.397218004 [W:onnxruntime:Default, scatter_nd.h:51 ScatterNDWithAtomicReduction] ScatterND with reduction=='none' only guarantees to be correct if indices are not duplicated.
2024-10-12 13:42:38.397267923 [W:onnxruntime:Default, scatter_nd.h:51 ScatterNDWithAtomicReduction] ScatterND with reduction=='none' only guarantees to be correct if indices are not duplicated.
2024-10-12 13:42:38.397321809 [W:onnxruntime:Default, scatter_nd.h:51 ScatterNDWithAtomicReduction] ScatterND with reduction=='none' only guarantees to be correct if indices are not duplicated.
2024-10-12 13:42:38.397371817 [W:onnxruntime:Default, scatter_nd.h:51 ScatterNDWithAtomicReduction] ScatterND with reduction=='none' only guarantees to be correct if indices are not duplicated.
2024-10-12 13:42:38.397416485 [W:onnxruntime:Default, scatter_nd.h:51 ScatterNDWithAtomicReduction] ScatterND with reduction=='none' only guarantees to be correct if indices are not duplicated.
2024-10-12 13:42:38.397464222 [W:onnxruntime:Default, scatter_nd.h:51 ScatterNDWithAtomicReduction] ScatterND with reduction=='none' only guarantees to be correct if indices are not duplicated.
2024-10-12 13:42:38.397510676 [W:onnxruntime:Default, scatter_nd.h:51 ScatterNDWithAtomicReduction] ScatterND with reduction=='none' only guarantees to be correct if indices are not duplicated.
2024-10-12 13:42:38.397558210 [W:onnxruntime:Default, scatter_nd.h:51 ScatterNDWithAtomicReduction] ScatterND with reduction=='none' only guarantees to be correct if indices are not duplicated.
2024-10-12 13:42:38.397601873 [W:onnxruntime:Default, scatter_nd.h:51 ScatterNDWithAtomicReduction] ScatterND with reduction=='none' only guarantees to be correct if indices are not duplicated.
2024-10-12 13:42:38.397654870 [W:onnxruntime:Default, scatter_nd.h:51 ScatterNDWithAtomicReduction] ScatterND with reduction=='none' only guarantees to be correct if indices are not duplicated.
2024-10-12 13:42:38.397698699 [W:onnxruntime:Default, scatter_nd.h:51 ScatterNDWithAtomicReduction] ScatterND with reduction=='none' only guarantees to be correct if indices are not duplicated.
2024-10-12 13:42:38.397750020 [W:onnxruntime:Default, scatter_nd.h:51 ScatterNDWithAtomicReduction] ScatterND with reduction=='none' only guarantees to be correct if indices are not duplicated.
2024-10-12 13:42:38.397796493 [W:onnxruntime:Default, scatter_nd.h:51 ScatterNDWithAtomicReduction] ScatterND with reduction=='none' only guarantees to be correct if indices are not duplicated.
2024-10-12 13:42:38.397844136 [W:onnxruntime:Default, scatter_nd.h:51 ScatterNDWithAtomicReduction] ScatterND with reduction=='none' only guarantees to be correct if indices are not duplicated.
2024-10-12 13:42:38.397892548 [W:onnxruntime:Default, scatter_nd.h:51 ScatterNDWithAtomicReduction] ScatterND with reduction=='none' only guarantees to be correct if indices are not duplicated.
2024-10-12 13:42:38.397932359 [W:onnxruntime:Default, scatter_nd.h:51 ScatterNDWithAtomicReduction] ScatterND with reduction=='none' only guarantees to be correct if indices are not duplicated.
2024-10-12 13:42:38.397978028 [W:onnxruntime:Default, scatter_nd.h:51 ScatterNDWithAtomicReduction] ScatterND with reduction=='none' only guarantees to be correct if indices are not duplicated.
2024-10-12 13:42:38.398022936 [W:onnxruntime:Default, scatter_nd.h:51 ScatterNDWithAtomicReduction] ScatterND with reduction=='none' only guarantees to be correct if indices are not duplicated.
2024-10-12 13:42:38.398067970 [W:onnxruntime:Default, scatter_nd.h:51 ScatterNDWithAtomicReduction] ScatterND with reduction=='none' only guarantees to be correct if indices are not duplicated.
2024-10-12 13:42:38.398112810 [W:onnxruntime:Default, scatter_nd.h:51 ScatterNDWithAtomicReduction] ScatterND with reduction=='none' only guarantees to be correct if indices are not duplicated.

2024-10-12 13:44:25.494281535 [E:onnxruntime:Default, cuda_call.cc:118 CudaCall] CUDNN failure 9: CUDNN_STATUS_NOT_SUPPORTED ; GPU=0 ; file=/onnxruntime_src/onnxruntime/core/providers/cuda/nn/pool.cc ; line=240 ; expr=PoolingForwardHelper(GetCudnnHandle(context), pooling_desc, &alpha, x_tensor, x_data, &beta, y_tensor, y_data); 
2024-10-12 13:44:25.494323991 [E:onnxruntime:, sequential_executor.cc:516 ExecuteKernel] Non-zero status code returned while running AveragePool node. Name:'/AveragePool_1' Status Message: CUDNN failure 9: CUDNN_STATUS_NOT_SUPPORTED ; GPU=0 ;  file=/onnxruntime_src/onnxruntime/core/providers/cuda/nn/pool.cc ; line=240 ; expr=PoolingForwardHelper(GetCudnnHandle(context), pooling_desc, &alpha, x_tensor, x_data, &beta, y_tensor, y_data);

python: 3.11.9
torch: 2.4.0+cu118
onnxruntime: 1.18.1

Oct 12 '24 06:10 qiangxinglin

Delete these assert lines may make warnings disappear.
I'm not familiar with onnx and I'm sorry that I can't help you.

Oct 14 '24 06:10 Windsrain