Add conversion from ONNX to PyTorch to allow cross conversion TensorFlow->ONNX->PyTorch

[valeriosofi]

At the moment we support only PyTorch to ONNX and TensorFlow to ONNX conversions. We could test and use this repo to convert an ONNX model to PyTorch in order to support TensorFlow to PyTorch conversion.

[SuperSecureHuman]

This also can be used to implement onnx to pytorch conversion (https://github.com/nebuly-ai/nebullvm/blob/main/nebullvm/operations/conversions/converters.py#L126)

Or is it implemented elsewhere?

[valeriosofi]

Hi @SuperSecureHuman, yep as you can see the pytorch_conversion method is not implemented for now, but we should test the repository linked above and if it works well we could use it to implement the method. If you want to contribute feel free to assign yourself the issue :)

[SuperSecureHuman]

I've started working (by directly importing the onnx2torch module). Not sure if I can assign myself to this because its gonna take me some time to understand the convertors api (and overall layout of all other apis).

Edit: I also have 0 experience with onnx :sweat_smile:

[valeriosofi]

Thanks, your help is very appreciated! Of course take your time to understand the code, and if you have any questions about how it works, just ask ;)

[SuperSecureHuman]

from nebullvm.operations.conversions.onnx import convert_onnx_to_torch
import onnx
onnx_model_path = '/home/venom/Downloads/mobilenetv2-12.onnx'
onnx_model = onnx.load(onnx_model_path)
output_file_path = '/home/venom/Downloads/model.pt'
device = 'cpu'
convert_onnx_to_torch(onnx_model, output_file_path, device)

For now the convertion works without error

Will try working further to improve it

Meanwhile, it would be great, if you could edit parts of my commit to make it more "module like" :)

[valeriosofi]

Great! Maybe check also with another couple of models to see if they are converted without errors as well. After that you could implement also the method pytorch_conversion in the TensorflowConverter class, where the model should be first converted to ONNX and then to PyTorch :)

[SuperSecureHuman]

Update: Sorry for the delay

import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3' 
from nebullvm.operations.conversions.onnx import convert_onnx_to_torch
import onnx
onnx_model_path = '/home/venom/Downloads/mobilenetv2-7.onnx'
onnx_model = onnx.load(onnx_model_path)
output_file_path = '/home/venom/Downloads/model.pt'
device = 'cpu'
outfile = convert_onnx_to_torch(onnx_model, output_file_path, device)
if outfile is not None:
    print("Converted successfully")
    print(outfile)
else:
    print("Conversion failed")

Now, added error handling

Commit - https://github.com/nebuly-ai/nebullvm/commit/cb74ee8baff286df796eca0e03069ae23c9251b2

[SuperSecureHuman]

onnx2torch module, has the requirement of opset version 9 (minimum)

In the above example, the model used opset7 (hence failed)

Trying to convert all the models in https://github.com/onnx/models

Open To view all the models from ONNX repo

Model	Status
bertsquad-10.onnx	Conversion failed
bertsquad-12-int8.onnx	Conversion failed
bertsquad-12.onnx	Conversion failed
bertsquad-8.onnx	Conversion failed
bidaf-9.onnx	Conversion failed
gpt2-10.onnx	Conversion failed
gpt2-lm-head-10.onnx	Conversion failed
gpt2-lm-head-bs-12.onnx	Conversion failed
roberta-base-11.onnx	Converted successfully
roberta-sequence-classification-9.onnx	Conversion failed
t5-decoder-with-lm-head-12.onnx	Converted successfully
t5-encoder-12.onnx	Converted successfully
age_googlenet.onnx	Converted successfully
gender_googlenet.onnx	Converted successfully
arcfaceresnet100-8.onnx	Conversion failed
emotion-ferplus-2.onnx	Conversion failed
emotion-ferplus-7.onnx	Conversion failed
emotion-ferplus-8.onnx	Conversion failed
version-RFB-320.onnx	Conversion failed
version-RFB-640.onnx	Conversion failed
bvlcalexnet-12-int8.onnx	Conversion failed
bvlcalexnet-12-qdq.onnx	Conversion failed
bvlcalexnet-12.onnx	Conversion failed
bvlcalexnet-3.onnx	Conversion failed
bvlcalexnet-6.onnx	Conversion failed
bvlcalexnet-7.onnx	Conversion failed
bvlcalexnet-8.onnx	Conversion failed
bvlcalexnet-9.onnx	Conversion failed
caffenet-12-int8.onnx	Conversion failed
caffenet-12-qdq.onnx	Conversion failed
caffenet-12.onnx	Conversion failed
caffenet-3.onnx	Conversion failed
caffenet-6.onnx	Conversion failed
caffenet-7.onnx	Conversion failed
caffenet-8.onnx	Conversion failed
caffenet-9.onnx	Conversion failed
densenet-12-int8.onnx	Conversion failed
densenet-12.onnx	Converted successfully
densenet-3.onnx	Conversion failed
densenet-6.onnx	Converted successfully
densenet-7.onnx	Conversion failed
densenet-8.onnx	Conversion failed
densenet-9.onnx	Converted successfully
efficientnet-lite4-11-int8.onnx	Conversion failed
efficientnet-lite4-11-qdq.onnx	Conversion failed
efficientnet-lite4-11.onnx	Converted successfully
googlenet-12-int8.onnx	Conversion failed
googlenet-12-qdq.onnx	Conversion failed
googlenet-12.onnx	Conversion failed
googlenet-3.onnx	Conversion failed
googlenet-6.onnx	Conversion failed
googlenet-7.onnx	Conversion failed
googlenet-8.onnx	Conversion failed
googlenet-9.onnx	Conversion failed
inception-v1-12-int8.onnx	Conversion failed
inception-v1-12-qdq.onnx	Conversion failed
inception-v1-12.onnx	Conversion failed
inception-v1-3.onnx	Conversion failed
inception-v1-6.onnx	Conversion failed
inception-v1-7.onnx	Conversion failed
inception-v1-8.onnx	Conversion failed
inception-v1-9.onnx	Conversion failed
inception-v2-3.onnx	Conversion failed
inception-v2-6.onnx	Conversion failed
inception-v2-7.onnx	Conversion failed
inception-v2-8.onnx	Conversion failed
inception-v2-9.onnx	Converted successfully
mnist-1.onnx	Conversion failed
mnist-12-int8.onnx	Conversion failed
mnist-12.onnx	Conversion failed
mnist-7.onnx	Conversion failed
mnist-8.onnx	Conversion failed
mobilenetv2-10.onnx	Converted successfully
mobilenetv2-12-int8.onnx	Conversion failed
mobilenetv2-12-qdq.onnx	Conversion failed
mobilenetv2-12.onnx	Converted successfully
mobilenetv2-7.onnx	Conversion failed
rcnn-ilsvrc13-3.onnx	Conversion failed
rcnn-ilsvrc13-6.onnx	Conversion failed
rcnn-ilsvrc13-7.onnx	Conversion failed
rcnn-ilsvrc13-8.onnx	Conversion failed
rcnn-ilsvrc13-9.onnx	Conversion failed
resnet101-v1-7.onnx	Conversion failed
resnet101-v2-7.onnx	Conversion failed
resnet152-v1-7.onnx	Conversion failed
resnet152-v2-7.onnx	Conversion failed
resnet18-v1-7.onnx	Conversion failed
resnet18-v2-7.onnx	Conversion failed
resnet34-v1-7.onnx	Conversion failed
resnet34-v2-7.onnx	Conversion failed
resnet50-caffe2-v1-3.onnx	Conversion failed
resnet50-caffe2-v1-6.onnx	Conversion failed
resnet50-caffe2-v1-7.onnx	Conversion failed
resnet50-caffe2-v1-8.onnx	Conversion failed
resnet50-caffe2-v1-9.onnx	Converted successfully
resnet50-v1-12-int8.onnx	Conversion failed
resnet50-v1-12-qdq.onnx	Conversion failed
resnet50-v1-12.onnx	Converted successfully
resnet50-v1-7.onnx	Conversion failed
resnet50-v2-7.onnx	Conversion failed
shufflenet-3.onnx	Conversion failed
shufflenet-6.onnx	Conversion failed
shufflenet-7.onnx	Conversion failed
shufflenet-8.onnx	Conversion failed
shufflenet-9.onnx	Converted successfully
shufflenet-v2-10.onnx	Conversion failed
shufflenet-v2-12-int8.onnx	Conversion failed
shufflenet-v2-12-qdq.onnx	Conversion failed
shufflenet-v2-12.onnx	Conversion failed
squeezenet1.0-12-int8.onnx	Conversion failed
squeezenet1.0-12.onnx	Conversion failed
squeezenet1.0-13-qdq.onnx	Conversion failed
squeezenet1.0-3.onnx	Conversion failed
squeezenet1.0-6.onnx	Conversion failed
squeezenet1.0-7.onnx	Conversion failed
squeezenet1.0-8.onnx	Conversion failed
squeezenet1.0-9.onnx	Conversion failed
squeezenet1.1-7.onnx	Conversion failed
vgg16-12-int8.onnx	Conversion failed
vgg16-12.onnx	Converted successfully
vgg16-7.onnx	Conversion failed
vgg16-bn-7.onnx	Conversion failed
vgg19-7.onnx	Conversion failed
vgg19-bn-7.onnx	Conversion failed
vgg19-caffe2-3.onnx	Conversion failed
vgg19-caffe2-6.onnx	Conversion failed
vgg19-caffe2-7.onnx	Conversion failed
vgg19-caffe2-8.onnx	Conversion failed
vgg19-caffe2-9.onnx	Conversion failed
zfnet512-12-int8.onnx	Conversion failed
zfnet512-12.onnx	Converted successfully
zfnet512-3.onnx	Conversion failed
zfnet512-6.onnx	Conversion failed
zfnet512-7.onnx	Conversion failed
zfnet512-8.onnx	Conversion failed
zfnet512-9.onnx	Converted successfully
ResNet101-DUC-12-int8.onnx	Conversion failed
ResNet101-DUC-12.onnx	Converted successfully
ResNet101-DUC-7.onnx	Conversion failed
FasterRCNN-10.onnx	Conversion failed
FasterRCNN-12-int8.onnx	Conversion failed
FasterRCNN-12.onnx	Conversion failed
fcn-resnet101-11.onnx	Converted successfully
fcn-resnet50-11.onnx	Converted successfully
fcn-resnet50-12-int8.onnx	Conversion failed
fcn-resnet50-12.onnx	Converted successfully
MaskRCNN-10.onnx	Conversion failed
MaskRCNN-12-int8.onnx	Conversion failed
MaskRCNN-12.onnx	Conversion failed
retinanet-9.onnx	Conversion failed
ssd-10.onnx	Conversion failed
ssd-12-int8.onnx	Conversion failed
ssd-12.onnx	Conversion failed
ssd_mobilenet_v1_10.onnx	Conversion failed
ssd_mobilenet_v1_12-int8.onnx	Conversion failed
ssd_mobilenet_v1_12.onnx	Conversion failed
tinyyolov2-7.onnx	Conversion failed
tinyyolov2-8.onnx	Conversion failed
tiny-yolov3-11.onnx	Conversion failed
yolov2-coco-9.onnx	Converted successfully
yolov3-10.onnx	Conversion failed
yolov3-12-int8.onnx	Conversion failed
yolov3-12.onnx	Conversion failed
yolov4.onnx	Conversion failed
candy-8.onnx	Conversion failed
candy-9.onnx	Conversion failed
mosaic-8.onnx	Conversion failed
mosaic-9.onnx	Conversion failed
pointilism-8.onnx	Conversion failed
pointilism-9.onnx	Conversion failed
rain-princess-8.onnx	Conversion failed
rain-princess-9.onnx	Conversion failed
udnie-8.onnx	Conversion failed
udnie-9.onnx	Conversion failed
super-resolution-10.onnx	Converted successfully

Filtering out non-quantised and models that match the min requirements

Open To view Non-quantised, opset 9+ models

model	status
bertsquad-10.onnx	Conversion failed
bertsquad-12.onnx	Conversion failed
bidaf-9.onnx	Conversion failed
gpt2-10.onnx	Conversion failed
gpt2-lm-head-10.onnx	Conversion failed
gpt2-lm-head-bs-12.onnx	Conversion failed
roberta-base-11.onnx	Converted successfully
roberta-sequence-classification-9.onnx	Conversion failed
t5-decoder-with-lm-head-12.onnx	Converted successfully
t5-encoder-12.onnx	Converted successfully
age_googlenet.onnx	Converted successfully
gender_googlenet.onnx	Converted successfully
version-RFB-320.onnx	Conversion failed
version-RFB-640.onnx	Conversion failed
bvlcalexnet-12.onnx	Conversion failed
bvlcalexnet-9.onnx	Conversion failed
caffenet-12.onnx	Conversion failed
caffenet-9.onnx	Conversion failed
densenet-12.onnx	Converted successfully
densenet-6.onnx	Converted successfully
densenet-9.onnx	Converted successfully
efficientnet-lite4-11.onnx	Converted successfully
googlenet-12.onnx	Conversion failed
googlenet-9.onnx	Conversion failed
inception-v1-12.onnx	Conversion failed
inception-v1-9.onnx	Conversion failed
inception-v2-9.onnx	Converted successfully
mnist-12.onnx	Conversion failed
mobilenetv2-10.onnx	Converted successfully
mobilenetv2-12.onnx	Converted successfully
rcnn-ilsvrc13-9.onnx	Conversion failed
resnet50-caffe2-v1-9.onnx	Converted successfully
resnet50-v1-12.onnx	Converted successfully
shufflenet-9.onnx	Converted successfully
shufflenet-v2-10.onnx	Conversion failed
shufflenet-v2-12.onnx	Conversion failed
squeezenet1.0-12-int8.onnx	Conversion failed
squeezenet1.0-12.onnx	Conversion failed
squeezenet1.0-9.onnx	Conversion failed
vgg16-12.onnx	Converted successfully
vgg19-caffe2-9.onnx	Conversion failed
zfnet512-12.onnx	Converted successfully
zfnet512-9.onnx	Converted successfully
ResNet101-DUC-12.onnx	Converted successfully
FasterRCNN-10.onnx	Conversion failed
FasterRCNN-12.onnx	Conversion failed
fcn-resnet101-11.onnx	Converted successfully
fcn-resnet50-11.onnx	Converted successfully
fcn-resnet50-12.onnx	Converted successfully
MaskRCNN-10.onnx	Conversion failed
MaskRCNN-12.onnx	Conversion failed
retinanet-9.onnx	Conversion failed
ssd-10.onnx	Conversion failed
ssd-12.onnx	Conversion failed
ssd_mobilenet_v1_10.onnx	Conversion failed
ssd_mobilenet_v1_12.onnx	Conversion failed
tiny-yolov3-11.onnx	Conversion failed
yolov2-coco-9.onnx	Converted successfully
yolov3-10.onnx	Conversion failed
yolov3-12.onnx	Conversion failed
yolov4.onnx	Conversion failed
candy-9.onnx	Conversion failed
mosaic-9.onnx	Conversion failed
pointilism-9.onnx	Conversion failed
rain-princess-9.onnx	Conversion failed
udnie-9.onnx	Conversion failed
super-resolution-10.onnx	Converted successfully

43 Failed, 24 Converted

[SuperSecureHuman]

Workaround for opset version

Onnx Version Conversion - Offical Docs

Example

import onnx
from onnx import version_converter
import torch
from onnx2torch import convert

# Load the ONNX model
model = onnx.load("model.onnx")
# Convert the model to the target version
target_version = 13
converted_model = version_converter.convert_version(model, target_version)
# Convert to torch
torch_model = convert(converted_model)
torch.save(torch_model, "model.pt")

This could be added to the docs, and in the error message