after using the BertOnnxModelTF.py to optimize the onnx bert model, only support batch_size==1?

[rigorosyang]

I use the BertOnnxModelTF to optimize the onnx bert model ,but running the inference occur below exception using batch_size > 1.

onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Non-zero status code returned while running SkipLayerNormalization node. Name:'SkipLayerNorm1' Status Message: skip is expected to have same shape as input

anyone can help me? Thanks in advance!

[tianleiwu]

@rigorosyang, have you resolved the problem?

I added a Jupyter notebook about Tensorflow Bert model optimization: Inference TensorFlow Bert Model with ONNX Runtime on CPU

[vgrabovets]

I'm facing similar issue. Described it here: https://github.com/microsoft/onnxruntime/issues/4980 There seems to be problem when you use 2 inputs (input_ids, attention_mask) instead of 3 (input_ids, attention_mask, token_type_ids).