Issue converting ONNX to TF2 SavedModel

[gcunhase]

TLDR; How can I convert an ONNX file into a TF2 SavedModel using onnx_tf?

Issue

I'm trying to load an exported model but it's loading as _UserObject instead of tf.keras.model:

import onnx
from onnx_tf.backend import prepare
import tensorflow as tf

# 1. Load onnx model
onnx_model = onnx.load(onnx_model_path)

# 2. Convert to TF
tf_rep = prepare(onnx_model)
saved_model_dir = onnx_model_path.split(".onnx")[0] + "_saved_model"
tf_rep.export_graph(saved_model_dir)

# 3. Load SavedModel
model = tf.keras.models.load_model(saved_model_dir)
input_shape = (1, 3, 800, 1280)
in_tensor = tf.random.normal(input_shape)
model(in_tensor)
print(model.summary())

Error:

WARNING:tensorflow:SavedModel saved prior to TF 2.5 detected when loading Keras model. Please ensure that you are saving the model with model.save() or tf.keras.models.save_model(), *NOT* tf.saved_model.save(). To confirm, there should be a file named "keras_metadata.pb" in the SavedModel directory.
Traceback (most recent call last):
  File "/home/nvidia/PycharmProjects/nvbugs/venv38_orin/lib/python3.8/site-packages/tensorflow/python/util/traceback_utils.py", line 153, in error_handler
    raise e.with_traceback(filtered_tb) from None
  File "/home/nvidia/PycharmProjects/nvbugs/venv38_orin/lib/python3.8/site-packages/tensorflow/python/saved_model/function_deserialization.py", line 289, in restored_function_body
    raise ValueError(
ValueError: Could not find matching concrete function to call loaded from the SavedModel. Got:
  Positional arguments (1 total):
    * Tensor("None_0:0", shape=(1, 3, 800, 1280), dtype=float32)
  Keyword arguments: {}

 Expected these arguments to match one of the following 1 option(s):

Option 1:
  Positional arguments (0 total):
    * 
  Keyword arguments: {'input_1': TensorSpec(shape=(None, 3, 800, 1280), dtype=tf.float32, name='input_1')}
python-BaseException

System

• Ubuntu 18.04
• Python 3.8.11
• Tensorflow 2.7.0
• onnx 1.10.2
• onnx-tf 1.9.0

[chinhuang007]

The saved model is a low level tf.saved_model which can be loaded using the low-level tf.saved_model API, https://www.tensorflow.org/api_docs/python/tf/saved_model. The high level keras model is currently not supported in onnx-tf converter.

[andreped]

The high level keras model is currently not supported in onnx-tf converter.

Would be a great addition. I believe most of the community that use TF only use it as backend with Keras.

[andreped]

@gcunhase I managed to get my model converted to Keras using: https://github.com/gmalivenko/onnx2keras

The original model was from pytorch, but I had challenges doing the final step from ONNX to Keras. This was done by:

from onnx2keras import onnx_to_keras
keras_model = onnx_to_keras(onnx_model, input_names=['input'], name_policy="renumerate")

Tested with tensorflow==2.8. In my case it was cruicial to change the name_policy to get it working.

I guess you could use this solution until proper keras support is added to onnx-tf.

[gcunhase]

@andreped thank you for your feedback. I'm getting the following error:

...
DEBUG:onnx2keras:... found all, continue
Traceback (most recent call last):
  File "/snap/pycharm-professional/280/plugins/python/helpers/pydev/pydevd.py", line 1491, in _exec
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "/snap/pycharm-professional/280/plugins/python/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "/home/nvidia/PycharmProjects/nvutils/onnx_utils/convert_onnx_to_tf.py", line 71, in <module>
    convert_onnx_to_keras(INPUT_PATH, OUTPUT_PATH + "_keras")
  File "/home/nvidia/PycharmProjects/nvutils/onnx_utils/convert_onnx_to_tf.py", line 60, in convert_onnx_to_keras
    keras_model = onnx_to_keras(onnx_model, input_names=input_names, name_policy="renumerate")
  File "/home/nvidia/PycharmProjects/nvutils/venv38/lib/python3.8/site-packages/onnx2keras/converter.py", line 175, in onnx_to_keras
    AVAILABLE_CONVERTERS[node_type](
  File "/home/nvidia/PycharmProjects/nvutils/venv38/lib/python3.8/site-packages/onnx2keras/padding_layers.py", line 19, in convert_padding
    params['mode'] = params['mode'].decode('ascii')
KeyError: 'mode'

It seems to be an issue with line 19 in onnx2keras/padding_layers.py: params['mode'] = params['mode'].decode('ascii'). The issue is that 'mode' is not a key in the params dictionary.

Any idea how to fix that?

I installed onnx2keras with:

pip install onnx2keras

My version is 0.0.24.

[andreped]

I did not have this issue. Where does this ONNX model originate from? In my case it was pytorch.

Assuming you converted to ONNX, are you certain ONNX conversion was successful. Do you get good predictions using the ONNX model? I have seen situations where ONNX conversion appeared to be successful, but when I wanted to actually use it in C++ through TensorRT/OpenVINO, I was unable to and now I was prompted errors.

Also, what is the reason for converting the ONNX model to Keras? In my case I wanted to finetune the model in my TF framework, and hence, it was important that I could add the model as part of the graph for backpropagation.

But seems like you got an issue when converting a constant layer, likely due to some layer doing some misc zero padding scheme in your model. Since you got an issue with the padding layer, which layers did you use to create your network? Could you share the details? Then I might be able to help, or at least debug the issue.

[gcunhase]

Thank you for your feedback!

The ONNX file I used was a simple ResNet50 model from TensorFlow. You can follow the steps below for a repro:

1. Install requirements:

pip install tensorflow-gpu==2.8.0
pip install nvidia-pyindex
pip install onnx-graphsurgeon
pip install git+https://github.com/onnx/tensorflow-onnx.git
pip install onnx2keras

2. Run script:

""" Converts ResNet50 to ONNX and back to Keras """

import onnx
import onnx_graphsurgeon as gs
import tensorflow as tf
import os
from tf2onnx import utils, convert

def convert_keras_to_onnx(keras_model, onnx_model_path):
    onnx_model_proto, _ = convert.from_keras(keras_model, opset=13)
    utils.save_protobuf(onnx_model_path, onnx_model_proto)

def convert_onnx_to_keras(input_path, output_path):
    # 1. Load onnx model
    onnx_model = onnx.load(input_path)
    graph = gs.import_onnx(onnx_model)
    input_names = [inp.name for inp in graph.inputs]

    # 2. Convert to Keras model
    from onnx2keras import onnx_to_keras
    keras_model = onnx_to_keras(onnx_model, input_names=input_names, name_policy="renumerate")
    keras_model.save(output_path)

def test_resnet50v1_workflow():
    root_path = "./resnet50"
    onnx_model_path_original = os.path.join(root_path, "original.onnx")

    # 1. Create ResNet50-v1 model
    input_img = tf.keras.layers.Input(shape=(224, 224, 3), name="input_1")
    model = tf.keras.applications.ResNet50(
        include_top=True,
        weights="imagenet",
        input_tensor=input_img,
        input_shape=None,
        pooling=None,
        classifier_activation="softmax",
    )

    # 2. Save original model as ONNX
    model.save(os.path.join(root_path, "saved_model_original"))
    convert_keras_to_onnx(model, onnx_model_path=onnx_model_path_original)

    # 3. Convert ONNX back to Keras
    convert_onnx_to_keras(onnx_model_path_original, os.path.join(root_path, "saved_model_recovered"))

if __name__ == '__main__':
    test_resnet50v1_workflow()

[andreped]

@gcunhase just tried this on my end using Python 3.8 and reproduced the error. I tried some permutations of potential solutions, but I always get the same issue.

Hence, I checked the onnx2keras repo, and went to issues. There are a lot of issues on KeyErrors. Essentially meaning operators that are not currently supported. For instance for the Tile operator: https://github.com/gmalivenko/onnx2keras/issues/116#issuecomment-814824548

Does not seem like these issues get resolved any time soon. Hence, I don't think you could use onnx2keras in your keras2onnx2keras workflow, sadly.

But it is rather strange. I successfully converted a ResNet18 model from pytorch -> ONNX -> Keras. So seems like there is something weird going on. I would suggest making an Issue at the onnx2keras repo to track this issue. But then again, they might be slow to solve this issue. Would be better if onnx-tf added support for Keras models.

BTW: Is there a reason why you are converting a keras model to onnx and back again?