optimum Add all available ONNX models to ORTConfigManager

This issue is linked to the ONNXConfig for all working group created for implementing an ONNXConfig for all available models. Let's extend our work and try to add all models with a fully functional ONNXConfig implemented to ORTConfigManager.

Adding models to ORTConfigManager will allow 🤗 Optimum users to boost even more their model with ONNX optimization capacity!

Feel free to join us in this adventure! Join the org by clicking here

Here is a non-exhaustive list of models that have one ONNXConfig and could be added to ORTConfigManager:

This includes only models with ONNXConfig implemented, if your target model doesn't have an ONNXConfig, please open an issue/or implement it (even cooler) in the 🤗 Transformers repository. Check this issue to know how to do

[x] Albert
[x] BART
[ ] BeiT
[x] BERT
[x] BigBird
[ ] BigBirdPegasus
[x] Blenderbot
[ ] BlenderbotSmall
[x] BLOOM
[x] CamemBERT
[ ] CLIP
[x] CodeGen
[ ] ConvNext
[ ] ConvBert
[ ] Data2VecText
[ ] Data2VecVision
[x] Deberta
[x] Deberta-v2
[ ] DeiT
[ ] DETR
[x] Distilbert
[x] ELECTRA
[ ] Flaubert
[x] GptBigCode
[x] GPT2
[x] GPTJ
[x] GPT-NEO
[x] GPT-NEOX
[ ] I-BERT
[ ] LayoutLM
[ ] LayoutLMv2
[ ] LayoutLMv3
[ ] LeViT
[x] Llama
[x] LongT5
[x] M2M100
[x] mBART
[x] MT5
[x] MarianMT
[ ] MobileBert
[ ] MobileViT
[x] nystromformer
[ ] OpenAIGPT-2
[ ] PLBart
[x] Pegasus
[ ] Perceiver
[ ] ResNet
[ ] RoFormer
[x] RoBERTa
[ ] SqueezeBERT
[x] T5
[x] ViT
[x] Whisper
[ ] XLM
[x] XLM-RoBERTa
[ ] XLM-RoBERTa-XL
[ ] YOLOS

If you want an example of implementation, I did one for MT5 #341.

You need to check how the attention_heads number and hidden_size arguments are named in the original implementation of your target model in the 🤗 Transformers source code. And then add it to the _conf dictionary. Finally, add your implemented model to tests to make it fully functional.

Aug 16 '22 08:08 chainyo

To update the list of supported models: BlenderBot, BLOOM, GptBigCode, GPT-NEOX, GPTJ, LongT5, Llama, mBART, M2M100, nystromformer, Pegasus,T5 ,ViT ,Whisper @michaelbenayoun @fxmarty is there still interest in advancing with other models?

Aug 11 '23 17:08 mszsorondo

Thanks @mszsorondo I just updated the list based on your comment 🙏 .

Aug 11 '23 19:08 chainyo

can i try to update for other model ?

Nov 19 '25 13:11 Rahuldrabit