Jingya HUANG

close as it turned to be too messy even after rebasing.

TF2ONNX offers the [support for exporting large ONNX](https://github.com/onnx/tensorflow-onnx/blob/v1.12.1/tf2onnx/convert.py#L427) tensors with external files, however by adding the flag to the ONNX exporter of transformers, it doesn't work correctly for the moment:...

> If there are no onnx-level solutions, it may be due to TF1 code (embeddings) in our models -- see [tensorflow/tensorflow#45041](https://github.com/tensorflow/tensorflow/issues/45041) > > Rewriting embeddings into TF2 code is in...

> Hey @ChainYo! I found some time and fixed the issues. Can we reopen? 😊 > > Adding support for the `global_attention_mask` was pretty easy after I tracked down the...

Hi @jcwchen, I tried setting `use_external_data_format=True` + `all_tensors_to_one_file=False` + `convert_attribute=True` with some smaller `size_threshold` (1024->512->256->128), it doesn't seems to solve the issue. Here is a screenshot of some files that...

Hi @jcwchen! By setting `size_threshold=0`, I got the same error message with only `model.onnx` exported. ``` Traceback (most recent call last): File "run_qa.py", line 544, in main() File "run_qa.py", line...

Hi @jcwchen, Thanks for taking care of this! However, by hacking `onnx=1.12.0` with your snippet, I still got the same error on the model size as before(also tried reducing the...

Hi @jcwchen, Thanks a lot for the debug details, it is really helpful! After several re-tries, unfortunately, I can still not reproduce the success. By changing the export options, the...

Hi @jcwchen, Thanks for the suggestion! I just found that actually the difference came from the fact that I was using optimization example instead of the quantization example. Now I...

P.S. This PR is based on the implementation of ORT inference for `ORTSeq2SeqTrainer`, it is built on the top of PR #189 .