Convert ONNX: Failed to export an ONNX attribute, since it's not constant, please try to make things (e.g., kernel size) static if possible

[mactiendinh]

I using torch 1.3.1, torchvision 0.4.0 'Failed to export an ONNX attribute, since it's not constant, please try to make things (e.g., kernel size) static if possible' at torch.onnx.export()

[asmitakhaneja]

This error occurs because of following code in model.py self.AdaptiveAvgPool = nn.AdaptiveAvgPool2d((None, 1)) If you change value from None to a static value(For example - 512) .This error will be resolved

[arshv27]

@asmitakhaneja Have you managed to port the code to ONNX? I have been stuck on this for a while, can I connect with you to resolve the same?

[asmitakhaneja]

@asmitakhaneja Have you managed to port the code to ONNX? I have been stuck on this for a while, can I connect with you to resolve the same?

@arshv27 yes, i have been able to export the model to ONNX Change self.AdaptiveAvgPool = nn.AdaptiveAvgPool2d((None, 1)) to self.AdaptiveAvgPool = nn.AdaptiveAvgPool2d((512, 1))

[arshv27]

@asmitakhaneja Have you managed to port the code to ONNX? I have been stuck on this for a while, can I connect with you to resolve the same?

@arshv27 yes, i have been able to export the model to ONNX Change self.AdaptiveAvgPool = nn.AdaptiveAvgPool2d((None, 1)) to self.AdaptiveAvgPool = nn.AdaptiveAvgPool2d((512, 1))

@asmitakhaneja What configuration are you using? I'm using TPS-Resnet-BiLSTM-Attn. Have you tried this?

[asmitakhaneja]

@arshv27 I am using None-Resnet-BiLSTM-CTC. Haven't yet tried for attention model

[arshv27]

@asmitakhaneja Thank you for your responses. Just a couple of more queries:-

1. Have you tried with the TPS module?
2. Is it possible to share your code for porting? It'll be a great help

[mactiendinh]

It looks like TPS module cannot be converted

[arshv27]

It looks like TPS module cannot be converted

@mactiendinh Are you able to identify the specific issue due to which TPS cannot be ported?

[asmitakhaneja]

It looks like TPS module cannot be converted

@mactiendinh Are you able to identify the specific issue due to which TPS cannot be ported?

yes, because TPS module has grid_sample which is not supported in ONNX

[arshv27]

Hey @asmitakhaneja and @mactiendinh . With your inputs, I was able to port my model in the configuration None-ResNet-BiLSTM-Attn. I changed used the weights for TPS-ResNet-BiLSTM-Attn and filtered out the keys of the TPS module, then reloaded the weights to my new model. I hope this approach works well?

My model seems to export successfully, but I'm unable to perform inference due to an error. I'm posting my code snipper for export as well as inference, which has mostly been taken from the documentation.

#ONNX CODE

dummy_output = model(image, text_for_pred, is_train=False)

onnxfile = "./onnx_models/TPS-ResNet-BiLSTM-Attn-case-sensitive.onnx"

torch.onnx.export(model, (image, text_for_pred), onnxfile, opset_version=12, verbose = True)

onnx_model = onnx.load(onnxfile)
onnx.checker.check_model(onnx_model)
onnx.helper.printable_graph(onnx_model.graph)

ort_session = onnxruntime.InferenceSession(onnxfile)

def to_numpy(tensor):
    return tensor.detach().cpu().numpy() if tensor.requires_grad else tensor.cpu().numpy()

# compute ONNX Runtime output prediction
ort_inputs = {ort_session.get_inputs()[0].name: (to_numpy(image)), ort_session.get_inputs()[1].name: to_numpy(text_for_pred)}
ort_outs = ort_session.run(None, ort_inputs)

# compare ONNX Runtime and PyTorch results
print(to_numpy(dummy_output).size, ort_outs[0].size, len(ort_outs))
np.testing.assert_allclose(to_numpy(dummy_output), ort_outs[0], rtol=1e-03, atol=1e-05)

print("Exported model has been tested with ONNXRuntime, and the result looks good!")

The error occurs at line ort_session = onnxruntime.InferenceSession(onnxfile)

onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Type Error: Type parameter (T) bound to different types (tensor(float) and tensor(int64) in node (ScatterElements_386).

Please let me know if you have some knowledge on this, whether I am making an error somewhere an if this highlights that my model doesn't export correctly

[zcswdt]

et me know if you have some knowledge on this, whether I am making an error somewhere an if this high

我也想转Onnx模型，请问大佬转成功了吗

[jusim]

Hey @arshv27, have you been able to export TPS-ResNet-BiLSTM-Attn?

[asmitakhaneja]

Hey @asmitakhaneja and @mactiendinh . With your inputs, I was able to port my model in the configuration None-ResNet-BiLSTM-Attn. I changed used the weights for TPS-ResNet-BiLSTM-Attn and filtered out the keys of the TPS module, then reloaded the weights to my new model. I hope this approach works well?

My model seems to export successfully, but I'm unable to perform inference due to an error. I'm posting my code snipper for export as well as inference, which has mostly been taken from the documentation.

#ONNX CODE

dummy_output = model(image, text_for_pred, is_train=False)

onnxfile = "./onnx_models/TPS-ResNet-BiLSTM-Attn-case-sensitive.onnx"

torch.onnx.export(model, (image, text_for_pred), onnxfile, opset_version=12, verbose = True)

onnx_model = onnx.load(onnxfile)
onnx.checker.check_model(onnx_model)
onnx.helper.printable_graph(onnx_model.graph)

ort_session = onnxruntime.InferenceSession(onnxfile)

def to_numpy(tensor):
    return tensor.detach().cpu().numpy() if tensor.requires_grad else tensor.cpu().numpy()

# compute ONNX Runtime output prediction
ort_inputs = {ort_session.get_inputs()[0].name: (to_numpy(image)), ort_session.get_inputs()[1].name: to_numpy(text_for_pred)}
ort_outs = ort_session.run(None, ort_inputs)

# compare ONNX Runtime and PyTorch results
print(to_numpy(dummy_output).size, ort_outs[0].size, len(ort_outs))
np.testing.assert_allclose(to_numpy(dummy_output), ort_outs[0], rtol=1e-03, atol=1e-05)

print("Exported model has been tested with ONNXRuntime, and the result looks good!")

The error occurs at line ort_session = onnxruntime.InferenceSession(onnxfile)

onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Type Error: Type parameter (T) bound to different types (tensor(float) and tensor(int64) in node (ScatterElements_386).

Please let me know if you have some knowledge on this, whether I am making an error somewhere an if this highlights that my model doesn't export correctly

@arshv27 With Attention module you cannot do the onnx conversion directly, because the module consists of loops and conditional statements for which you will have to dig into the concept of tracing and scripting which is mentioned in FAq section here: https://pytorch.org/docs/stable/onnx.html#operator-export-type

[asmitakhaneja]

et me know if you have some knowledge on this, whether I am making an error somewhere an if this high

I also want to transfer the Onnx model. Is the transfer successful?

@zcswdt It has been successful for None-Resnet-BiLSTM-CTC.pth

[zcswdt]

et me know if you have some knowledge on this, whether I am making an error somewhere an if this high

I also want to transfer the Onnx model. Is the transfer successful?

@zcswdt It has been successful for None-Resnet-BiLSTM-CTC.pth

Please tell me, how did you do it? Thank you

[ziyeZzz]

et me know if you have some knowledge on this, whether I am making an error somewhere an if this high

I also want to transfer the Onnx model. Is the transfer successful?

@zcswdt It has been successful for None-Resnet-BiLSTM-CTC.pth

Please tell me, how did you do it? Thank you

Hi, Have you transfered successfully?

[ziyeZzz]

If modified nn.AdaptiveAvgPool2d((None, 1)) to nn.AdaptiveAvgPool2d((512, 1)), is it means that we need to train the model again?

[kishcs]

I am getting error as below:

model.load_state_dict(torch.load(trained_model, map_location=device))
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1604, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for Model:
        Missing key(s) in state_dict: "Transformation.LocalizationNetwork.conv.0.weight", "Transformation.LocalizationNetwork.conv.1.weight", "Transformation.LocalizationNetwork.conv.1.bias", "Transformation.LocalizationNetwork.conv.1.running_mean",