Tianlei Wu

@[nonstopfor](https://github.com/nonstopfor), you can change dynamic_axes to fixed shape with onnx python API like the following: ``` import onnx model = onnx.load("input.onnx") for tensor in model.graph.input: for dim_proto in tensor.type.tensor_type.shape.dim: if...

@[wadhwasahil](https://github.com/wadhwasahil), try the following: ``` for tensor in model.graph.input: for dim_proto in tensor.type.tensor_type.shape.dim: if dim_proto.HasField("dim_param"): # and dim_proto.dim_param == 'UNK_590': dim_proto.Clear() dim_proto.dim_value = 4 # fixed batch size for tensor...

@[jvivo](https://github.com/jvivo), example script to convert onnx graph inputs to int32: https://github.com/microsoft/onnxruntime/blob/c6732c079b4fbf644bc939729c87d70195251d83/onnxruntime/python/tools/transformers/onnx_model_bert.py#L125-L202 Another way is to use int32 tensors as input to export pytorch model to onnx.

@rigorosyang, have you resolved the problem? I added a Jupyter notebook about Tensorflow Bert model optimization: [Inference TensorFlow Bert Model with ONNX Runtime on CPU](https://github.com/microsoft/onnxruntime/blob/master/onnxruntime/python/tools/bert/notebooks/Tensorflow_Keras_Bert-Squad_OnnxRuntime_CPU.ipynb)

@agregory96, thanks for the comment. I have a notebook for end to end run: [Inference TensorFlow Bert Model with ONNX Runtime on CPU](https://github.com/microsoft/onnxruntime/blob/master/onnxruntime/python/tools/bert/notebooks/Tensorflow_Keras_Bert-Squad_OnnxRuntime_CPU.ipynb). You can follow [this notebook](https://github.com/microsoft/onnxruntime/blob/master/onnxruntime/python/tools/bert/notebooks/PyTorch_Bert-Squad_OnnxRuntime_GPU.ipynb) for OnnxRuntime...

It does not reproduce in my machine. I used latest nightly version on python 3.8 and Ubuntu 18.04: PyTorch: 1.13.0.dev20220830+cu113 from `pip3 install --pre torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/nightly/cu113` ort-nightly-gpu:...

Possible caused by --enable_training. In my test, I did not enable training.

@AllenTiTaiWang, thanks for the effort. Could you add license header to py files? Did you able to test the accuracy of exported model (like compare the generation result with pytorch...

I saw different results from PyTorch and ORT. For example, the stdout from `python export.py -m facebook/bart-base`. I used PyTorch 1.12.1+cu116, transformers 4.18.0 and onnxruntime-gpu 1.12.1: ``` pytorch inference ......

I think some operator supports bfloat16. To find which operator supports, search "bfloat16" in https://github.com/microsoft/onnxruntime/blob/454f77cd94901cb95c92b20c60565408b2be045c/docs/OperatorKernels.md @ildoonet, Which operators (or let us know which model) do you want add support of...