tf2onnx.convert.from_keras Produces Non-Deterministic ONNX Graph

[ceruleangu]

Problem We tried to convert a TensorFlow (or Keras) Bert model into ONNX model, but every time we run with the same piece of code (below) the ONNX graphs are a little different in node names. But we want the conversion to be deterministic, i.e., the converted ONNX node names are exactly the same each time we run. We think this https://github.com/onnx/tensorflow-onnx/issues/1789 is a related problem but we do not see a resolution?

To Reproduce

import tensorflow as tf
import tf2onnx
from transformers import TFBertModel

model = TFBertModel.from_pretrained("bert-base-cased")
spec = [tf.TensorSpec(shape=[1, 128], dtype=tf.int32, name='input_ids'), 
        tf.TensorSpec(shape=[1, 128], dtype=tf.int32, name='token_type_ids'),  
        tf.TensorSpec(shape=[1, 128], dtype=tf.int32, name='attention_mask')]
onnx_model, _ = tf2onnx.convert.from_keras(model, input_signature=spec, opset=15)

print(onnx_model.graph.node)

System information

• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux
• Tensorflow Version: 2.7.0
• Python version: 3.7.6

Screenshots Some of the example differences:

Additional context We suspect ONNX is optimizing the graph (constant folding, etc) as described here: https://huggingface.co/docs/transformers/v4.14.1/en/serialization#optimizations The optimization step is bringing the different names maybe due to some global state but we are not sure.

[fatcat-z]

These optimizations happened within tf2onnx, probably this is something we can improve to generate a consistent output for such model.