I converted pytorch model to onnx, but failed to run onnx.

[ccl-private]

Here is my code. `import torch import torch.onnx from retinanet import model

model= model.resnet50(num_classes=80,) model.load_state_dict(torch.load('../models/coco_resnet_50_map_0_335_state_dict.pt')) model=model.cuda() dummy_input = torch.randn(1, 3, 608, 1024, device='cuda') torch.onnx.export(model, dummy_input, "onnx_model_name.onnx", opset_version=11)

import onnx

model = onnx.load("onnx_model_name.onnx")

onnx.checker.check_model(model)

print(onnx.helper.printable_graph(model.graph))

import onnxruntime as ort import numpy as np

ort_session = ort.InferenceSession('onnx_model_name.onnx') input_name = ort_session.get_inputs()[0].name # 'data' out = ort_session.get_outputs() outputs = list() for i in out: outputs.append(i.name) print(input_name) print(outputs)

outputs = ort_session.run(outputs, input_feed={input_name: np.random.randn(1, 3, 608, 1024).astype(np.float32)})

print(outputs[0])`

[ccl-private]

Here is the error log: onnxruntime.capi.onnxruntime_pybind11_state.RuntimeException: [ONNXRuntimeError] : 6 : RUNTIME_EXCEPTION : Non-zero status code returned while running Concat node. Name:'Concat_711' Status Message: /onnxruntime_src/onnxruntime/core/providers/cpu/tensor/concat.cc:72 onnxruntime::common::Status onnxruntime::ConcatBase::PrepareForCompute(onnxruntime::OpKernelContext*, const std::vector<const onnxruntime::Tensor*>&, onnxruntime::Prepare&) const inputs_n_rank == inputs_0_rank was false. Ranks of input data are different, cannot concatenate them. expected rank: 1 got: 2

[ccl-private]

can someone try conversion and run onnx?

[Sandeep418]

@ccl-private check this one to convert torch to onnx. https://github.com/onnx/models/blob/master/vision/object_detection_segmentation/retinanet/dependencies/retinanet-export.py

[dwaithe]

I had the same problem. It was the last concatenation: finalAnchorBoxesCoordinates = torch.cat((finalAnchorBoxesCoordinates, anchorBoxes[anchors_nms_idx])) Unlike the other concatenations onnx doesn't like it as anchorBoxes is 2-D, the others are 1-D. It's only the first loop of the for loop (i.e. i=0). Therefore you can initialise finalAnchorBoxesCoordinates as anchorBoxes for i =0 and then concatenate from there: if i ==0: finalAnchorBoxesCoordinates = anchorBoxes[anchors_nms_idx] else: finalAnchorBoxesCoordinates = torch.cat((finalAnchorBoxesCoordinates, anchorBoxes[anchors_nms_idx]))

Worked for me.

[satpalsr]

Any of you successfully convert to onnx?

I get these warnings.

TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  image_shape = np.array(image_shape)
pytorch-retinanet/retinanet/anchors.py:38: TracerWarning: torch.from_numpy results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
  return torch.from_numpy(all_anchors.astype(np.float32)).cuda()
pytorch-retinanet/retinanet/model.py:268: TracerWarning: torch.Tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
  finalScores = torch.Tensor([])
pytorch-retinanet/retinanet/model.py:269: TracerWarning: torch.Tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
  finalAnchorBoxesIndexes = torch.Tensor([]).long()
pytorch-retinanet/retinanet/model.py:270: TracerWarning: torch.Tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
  finalAnchorBoxesCoordinates = torch.Tensor([])
pytorch-retinanet/retinanet/model.py:280: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if scores_over_thresh.sum() == 0:
Traceback (most recent call last):
  File "torch2onnx.py", line 8, in <module>
    torch.onnx.export(model, dummy_input, 'model.onnx',verbose=False, input_names=input_names, output_names=output_names, export_params=True,  opset_version=11)
  File "pytorch-retinanet/env/lib/python3.6/site-packages/torch/onnx/__init__.py", line 320, in export
    custom_opsets, enable_onnx_checker, use_external_data_format)
  File "pytorch-retinanet/env/lib/python3.6/site-packages/torch/onnx/utils.py", line 111, in export
    custom_opsets=custom_opsets, use_external_data_format=use_external_data_format)
  File "pytorch-retinanet/env/lib/python3.6/site-packages/torch/onnx/utils.py", line 729, in _export
    dynamic_axes=dynamic_axes)
  File "pytorch-retinanet/env/lib/python3.6/site-packages/torch/onnx/utils.py", line 501, in _model_to_graph
    module=module)
  File "pytorch-retinanet/env/lib/python3.6/site-packages/torch/onnx/utils.py", line 215, in _optimize_graph
    torch._C._jit_pass_onnx_set_dynamic_input_shape(graph, dynamic_axes, input_names)
TypeError: _jit_pass_onnx_set_dynamic_input_shape(): incompatible function arguments. The following argument types are supported:
    1. (arg0: torch::jit::Graph, arg1: Dict[str, Dict[int, str]], arg2: List[str]) -> None