fxmarty

Thank you @kirayomato, I remember noticing large variability there as well. The relevant code is here: https://github.com/huggingface/optimum/blob/0f0a66303425b476bd5e209c076419a404238bb3/tests/benchmark/benchmark_gptq.py#L306-L329 https://github.com/huggingface/optimum/blob/0f0a66303425b476bd5e209c076419a404238bb3/tests/benchmark/benchmark_gptq.py#L354-L355 Does something look wrong to you there?

@xenova Can you push to this branch so that the CI is rerun?

Hi @alachyan-ml, thank you for the report! I run ``` optimum-cli export onnx -m openai/clip-vit-base-patch32 --framework pt clip_onnx ``` with ``` torch==2.1.0 optimum==1.13.2 transformers==4.34.0 ``` and can not reproduce this...

Hi @isaac-chung, I used the model id `openai/clip-vit-base-patch32` shared by @alachyan-ml: https://github.com/huggingface/optimum/issues/1450#issue-1943412047 Let me have a look at `laion/CLIP-ViT-B-32-laion2B-s34B-b79K` and get back to you.

For sure, thank you @isaac-chung!

Do CPUs have a int4 multiply-add opset?

The issue exists also for `torch==2.2.0.dev20231102+cu118`, `optimum==1.13.2`, `transformers==4.34.1`

Hi for reference we are upstreaming SDPA in Transformers, maybe it would be a better fit for longT5: https://github.com/huggingface/transformers/issues/28005 Leaving this open as we may leverage nested tensors for longt5...

@ppanchad-amd Just tested, this is fixed thank you.

Hi @sayakpaul , can you show what are the external data generated by tf2onnx? It looks like the model is invalid / the path to the external data can't be...