fxmarty
fxmarty
Hi, `THUDM/chatglm3-6b` is not a Transformers model and the export is expected to fail. Could you share your export log here?
@Harini-Vemula-2382 Thank you. Llava ONNX export is not yet supported. A PR is open: https://github.com/huggingface/optimum/pull/1790 The error you get is likely ``` File "/home/felix/transformers/src/transformers/models/auto/auto_factory.py", line 566, in from_pretrained raise ValueError(...
@pradeepdev-1995 Thank you for the request. You can give a try to `pip install optimum[exporters-tf]` and ``` optimum-cli export onnx --model /path/to/your/model --framework tf onnx_output/ ``` should (hopefully) work. You...
About 1., can you try: `pip uninstall optimum && pip install optimum[exporters-tf]`. What are the install logs? About 2., I'll try to reproduce, thank you.
Hi @vjsrinivas, feel free to open a PR, happy to help there.
@nickrwann @jacob-vincent-mink I can not reproduce the issue, using ``` optimum-cli export onnx --model google/gemma-2b-it gemma_onnx_with_past ``` with the environment: ``` optimum==d87efb25c98741501fbf6da0d270fc181611b795 transformers==d47966536cd5ac1ed7e140edac65f00f471f656f torch==2.2.1 python==3.10.13 tokenizers==0.15.2 onnx==1.15.0 onnxruntime==1.17.1 accelerate not...
> It’s worth noting that the ONNX model I’m having trouble with was converted with the optimum-cli - does the CLI also perform the call to eval() before/during conversion? Yes:...
@jacob-vincent-mink thank you, next week I'll have a windows laptop at hand, I can have a look at whether the bug is in Pytorch, ORT, or Optimum. Related: https://github.com/huggingface/optimum/issues/1310
Thank you @mgiessing! It is possible that the ONNX model is valid, but ORT is [missing](https://github.com/onnx/onnx/blob/main/docs/Operators.md#where) some operators for bf16. It can halso be a bug, I will have a...
@mgiessing `Where` (used in https://github.com/huggingface/transformers/blob/caa5c65db1f4db617cdac2ad667ba62edf94dd98/src/transformers/models/llama/modeling_llama.py#L1086) is not implemented for BF16 dtype in ORT https://github.com/microsoft/onnxruntime/blob/v1.17.1/docs/OperatorKernels.md However it is valid in ONNX standard: https://github.com/onnx/onnx/blob/main/docs/Operators.md#where I suggest you to open a feature request...