Cannot infer shape for StatefulPartitionedCall/model_2/movie_index_lookup/None_Lookup/LookupTableFindV2:

[thongvhoang]

Describe the bug

I am trying to convert the BST model for recommendation to onnx format. Even though the message is "Warning - Cannot infer", the onnx model is successful and load to Redis AI successfully. When running the conversion, I get an error message: !python -m tf2onnx.convert --saved-model '/home/devops/lemuck-airflow-etl_1/dags/scripts/transformer.tf' --output '/home/devops/lemuck-airflow-etl_1/dags/scripts/tensor.tf/transformer_saved_opset_16.onnx' --opset 18

2023-04-04 15:58:25.848632: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
/home/devops/anaconda3/envs/transformer_recommendation/lib/python3.8/runpy.py:127: RuntimeWarning: 'tf2onnx.convert' found in sys.modules after import of package 'tf2onnx', but prior to execution of 'tf2onnx.convert'; this may result in unpredictable behaviour
  warn(RuntimeWarning(msg))
2023-04-04 15:58:26,520 - WARNING - '--tag' not specified for saved_model. Using --tag serve
2023-04-04 15:58:27,542 - INFO - Signatures found in model: [serving_default].
2023-04-04 15:58:27,542 - WARNING - '--signature_def' not specified, using first signature: serving_default
2023-04-04 15:58:27,542 - INFO - Output names: ['dense_11']
2023-04-04 15:58:27,943 - INFO - Using tensorflow=2.12.0, onnx=1.13.1, tf2onnx=1.14.0/8f8d49
2023-04-04 15:58:27,943 - INFO - Using opset <onnx, 18>
2023-04-04 15:58:27,993 - WARNING - Cannot infer shape for StatefulPartitionedCall/model_2/movie_index_lookup/None_Lookup/LookupTableFindV2: StatefulPartitionedCall/model_2/movie_index_lookup/None_Lookup/LookupTableFindV2:0
2023-04-04 15:58:27,993 - WARNING - Cannot infer shape for StatefulPartitionedCall/model_2/string_lookup_4/None_Lookup/LookupTableFindV2: StatefulPartitionedCall/model_2/string_lookup_4/None_Lookup/LookupTableFindV2:0
2023-04-04 15:58:27,993 - WARNING - Cannot infer shape for StatefulPartitionedCall/model_2/string_lookup_5/None_Lookup/LookupTableFindV2: StatefulPartitionedCall/model_2/string_lookup_5/None_Lookup/LookupTableFindV2:0
2023-04-04 15:58:27,993 - WARNING - Cannot infer shape for StatefulPartitionedCall/model_2/string_lookup_6/None_Lookup/LookupTableFindV2: StatefulPartitionedCall/model_2/string_lookup_6/None_Lookup/LookupTableFindV2:0
2023-04-04 15:58:27,993 - WARNING - Cannot infer shape for StatefulPartitionedCall/model_2/string_lookup_7/None_Lookup/LookupTableFindV2: StatefulPartitionedCall/model_2/string_lookup_7/None_Lookup/LookupTableFindV2:0
2023-04-04 15:58:27,993 - WARNING - Cannot infer shape for StatefulPartitionedCall/model_2/movie_index_lookup/Equal: StatefulPartitionedCall/model_2/movie_index_lookup/Equal:0
2023-04-04 15:58:27,993 - WARNING - Cannot infer shape for StatefulPartitionedCall/model_2/movie_index_lookup/None_Lookup_1/LookupTableFindV2: StatefulPartitionedCall/model_2/movie_index_lookup/None_Lookup_1/LookupTableFindV2:0
2023-04-04 15:58:27,993 - WARNING - Cannot infer shape for StatefulPartitionedCall/model_2/string_lookup_4/Equal: StatefulPartitionedCall/model_2/string_lookup_4/Equal:0
2023-04-04 15:58:27,993 - WARNING - Cannot infer shape for StatefulPartitionedCall/model_2/string_lookup_5/Equal: StatefulPartitionedCall/model_2/string_lookup_5/Equal:0
2023-04-04 15:58:27,993 - WARNING - Cannot infer shape for StatefulPartitionedCall/model_2/string_lookup_6/Equal: StatefulPartitionedCall/model_2/string_lookup_6/Equal:0
2023-04-04 15:58:27,993 - WARNING - Cannot infer shape for StatefulPartitionedCall/model_2/string_lookup_7/Equal: StatefulPartitionedCall/model_2/string_lookup_7/Equal:0
2023-04-04 15:58:27,993 - WARNING - Cannot infer shape for StatefulPartitionedCall/model_2/movie_index_lookup/Equal_2: StatefulPartitionedCall/model_2/movie_index_lookup/Equal_2:0
2023-04-04 15:58:27,993 - WARNING - Cannot infer shape for StatefulPartitionedCall/model_2/movie_index_lookup/GatherNd: StatefulPartitionedCall/model_2/movie_index_lookup/GatherNd:0
2023-04-04 15:58:27,993 - WARNING - Cannot infer shape for StatefulPartitionedCall/model_2/string_lookup_4/GatherNd: StatefulPartitionedCall/model_2/string_lookup_4/GatherNd:0
2023-04-04 15:58:27,993 - WARNING - Cannot infer shape for StatefulPartitionedCall/model_2/string_lookup_5/GatherNd: StatefulPartitionedCall/model_2/string_lookup_5/GatherNd:0
2023-04-04 15:58:27,993 - WARNING - Cannot infer shape for StatefulPartitionedCall/model_2/string_lookup_6/GatherNd: StatefulPartitionedCall/model_2/string_lookup_6/GatherNd:0
2023-04-04 15:58:27,993 - WARNING - Cannot infer shape for StatefulPartitionedCall/model_2/string_lookup_7/GatherNd: StatefulPartitionedCall/model_2/string_lookup_7/GatherNd:0
2023-04-04 15:58:27,993 - WARNING - Cannot infer shape for StatefulPartitionedCall/model_2/movie_index_lookup/GatherNd_1: StatefulPartitionedCall/model_2/movie_index_lookup/GatherNd_1:0
2023-04-04 15:58:27,993 - WARNING - Cannot infer shape for StatefulPartitionedCall/model_2/movie_embedding/embedding_lookup: StatefulPartitionedCall/model_2/movie_embedding/embedding_lookup:0
2023-04-04 15:58:27,993 - WARNING - Cannot infer shape for StatefulPartitionedCall/model_2/genres_vector/embedding_lookup: StatefulPartitionedCall/model_2/genres_vector/embedding_lookup:0
2023-04-04 15:58:27,993 - WARNING - Cannot infer shape for StatefulPartitionedCall/model_2/concatenate_8/concat: StatefulPartitionedCall/model_2/concatenate_8/concat:0
2023-04-04 15:58:27,993 - WARNING - Cannot infer shape for StatefulPartitionedCall/model_2/movie_embedding/embedding_lookup_1: StatefulPartitionedCall/model_2/movie_embedding/embedding_lookup_1:0
2023-04-04 15:58:27,993 - WARNING - Cannot infer shape for StatefulPartitionedCall/model_2/genres_vector/embedding_lookup_1: StatefulPartitionedCall/model_2/genres_vector/embedding_lookup_1:0
2023-04-04 15:58:27,993 - WARNING - Cannot infer shape for StatefulPartitionedCall/model_2/occupation_embedding/embedding_lookup: StatefulPartitionedCall/model_2/occupation_embedding/embedding_lookup:0
2023-04-04 15:58:27,993 - WARNING - Cannot infer shape for StatefulPartitionedCall/model_2/concatenate_7/concat: StatefulPartitionedCall/model_2/concatenate_7/concat:0
2023-04-04 15:58:27,994 - WARNING - Cannot infer shape for StatefulPartitionedCall/model_2/age_group_embedding/embedding_lookup: StatefulPartitionedCall/model_2/age_group_embedding/embedding_lookup:0
2023-04-04 15:58:27,994 - WARNING - Cannot infer shape for StatefulPartitionedCall/model_2/sex_embedding/embedding_lookup: StatefulPartitionedCall/model_2/sex_embedding/embedding_lookup:0
2023-04-04 15:58:27,994 - WARNING - Cannot infer shape for StatefulPartitionedCall/model_2/user_id_embedding/embedding_lookup: StatefulPartitionedCall/model_2/user_id_embedding/embedding_lookup:0
2023-04-04 15:58:27,994 - WARNING - Cannot infer shape for StatefulPartitionedCall/model_2/concatenate_6/concat: StatefulPartitionedCall/model_2/concatenate_6/concat:0
2023-04-04 15:58:28,007 - INFO - Computed 2 values for constant folding
2023-04-04 15:58:28,056 - INFO - folding node using tf type=StridedSlice, name=StatefulPartitionedCall/model_2/layer_normalization_4/strided_slice_2
2023-04-04 15:58:28,056 - INFO - folding node using tf type=StridedSlice, name=StatefulPartitionedCall/model_2/layer_normalization_5/strided_slice_2
2023-04-04 15:58:28,139 - INFO - Optimizing ONNX model
2023-04-04 15:58:28,598 - INFO - replacing einsum node 'StatefulPartitionedCall/model_2/multi_head_attention_2/value/einsum/Einsum' by its decomposed version, name of the last node 'Identity__253'.
2023-04-04 15:58:28,888 - INFO - replacing einsum node 'StatefulPartitionedCall/model_2/multi_head_attention_2/attention_output/einsum/Einsum' by its decomposed version, name of the last node 'Identity__294'.
2023-04-04 15:58:29,120 - INFO - replacing einsum node 'StatefulPartitionedCall/model_2/multi_head_attention_2/einsum_1/Einsum' by its decomposed version, name of the last node 'Identity__353'.
2023-04-04 15:58:29,124 - INFO - replacing einsum node 'StatefulPartitionedCall/model_2/multi_head_attention_2/query/einsum/Einsum' by its decomposed version, name of the last node 'Identity__392'.
2023-04-04 15:58:29,332 - INFO - replacing einsum node 'StatefulPartitionedCall/model_2/multi_head_attention_2/einsum/Einsum' by its decomposed version, name of the last node 'Identity__450'.
2023-04-04 15:58:29,336 - INFO - replacing einsum node 'StatefulPartitionedCall/model_2/multi_head_attention_2/key/einsum/Einsum' by its decomposed version, name of the last node 'Identity__489'.
2023-04-04 15:58:30,158 - INFO - After optimization: Cast -21 (39->18), Concat +9 (14->23), Const -120 (177->57), Einsum -6 (6->0), Gather +19 (16->35), Gemm +4 (0->4), Identity -2 (2->0), MatMul +2 (6->8), Max +2 (0->2), ReduceProd +4 (8->12), Reshape +4 (21->25), Shape +10 (8->18), Squeeze +1 (11->12), Transpose +7 (0->7), Unsqueeze -12 (25->13)
2023-04-04 15:58:30,173 - INFO - 
2023-04-04 15:58:30,173 - INFO - Successfully converted TensorFlow model /home/devops/lemuck-airflow-etl_1/dags/scripts/transformer.tf to ONNX
2023-04-04 15:58:30,173 - INFO - Model inputs: ['age_group', 'occupation', 'sequence_movie_ids', 'sequence_ratings', 'sex', 'target_movie_id', 'user_id']
2023-04-04 15:58:30,173 - INFO - Model outputs: ['dense_11']
2023-04-04 15:58:30,173 - INFO - ONNX model is saved at /home/devops/lemuck-airflow-etl_1/dags/scripts/tensor.tf/transformer_saved_opset_16.onnx

Urgency

System information

• OS Platform and Distribution: Linux Ubuntu 22.04
• TensorFlow Version: 2.12.0
• Python version: 3.8.16
• TensorflowOnnx : tf2onnx=1.14.0/8f8d49

To Reproduce

!python -m tf2onnx.convert --saved-model '/home/devops/lemuck-airflow-etl_1/dags/scripts/transformer.tf' --output '/home/devops/lemuck-airflow-etl_1/dags/scripts/tensor.tf/transformer_saved_opset_16.onnx' --opset 18

Model : transformer.tf.zip

Screenshots Additional context Question :

• How to fix this bug ?
• What does the warning affects to the model ? Thank you very much

Further information Model: BST model Model format: tf Model opset: 14-18 (see conversion command above, but also happens with other opset values) transformer_saved_opset_16.zip