Compatible with custom models

[QIN2DIM]

A very helpful project. 🦾

Does the author have any recent ideas for updating the project?

I hope to add some features, if you are interested to try (I can PR this part of the code):

1. OPEN-CLIP Model Zoo

   I noticed that you are only using clip to export pre-trained models. And you are limiting the model names that can be manipulated through the allows_list.

   In fact, it only takes a few lines of code to import the latest updates to the OPEN CLIP model.

2. ONNX simplify

3. INT8 Quant

---

for tag in open_clip.list_models():
    m = open_clip.list_pretrained_tags_by_model(tag)
    print(tag, m)

coca_base []
coca_roberta-ViT-B-32 []
coca_ViT-B-32 ['laion2b_s13b_b90k', 'mscoco_finetuned_laion2b_s13b_b90k']
coca_ViT-L-14 ['laion2b_s13b_b90k', 'mscoco_finetuned_laion2b_s13b_b90k']
convnext_base ['laion400m_s13b_b51k']
convnext_base_w ['laion2b_s13b_b82k', 'laion2b_s13b_b82k_augreg', 'laion_aesthetic_s13b_b82k']
convnext_base_w_320 ['laion_aesthetic_s13b_b82k', 'laion_aesthetic_s13b_b82k_augreg']
convnext_large []
convnext_large_d ['laion2b_s26b_b102k_augreg']
convnext_large_d_320 ['laion2b_s29b_b131k_ft', 'laion2b_s29b_b131k_ft_soup']
convnext_small []
convnext_tiny []
convnext_xlarge []
convnext_xxlarge ['laion2b_s34b_b82k_augreg', 'laion2b_s34b_b82k_augreg_rewind', 'laion2b_s34b_b82k_augreg_soup']
convnext_xxlarge_320 []
EVA01-g-14 ['laion400m_s11b_b41k']
EVA01-g-14-plus ['merged2b_s11b_b114k']
EVA02-B-16 ['merged2b_s8b_b131k']
EVA02-E-14 ['laion2b_s4b_b115k']
EVA02-E-14-plus ['laion2b_s9b_b144k']
EVA02-L-14 ['merged2b_s4b_b131k']
EVA02-L-14-336 ['merged2b_s6b_b61k']
mt5-base-ViT-B-32 []
mt5-xl-ViT-H-14 []
RN50 ['openai', 'yfcc15m', 'cc12m']
RN50-quickgelu ['openai', 'yfcc15m', 'cc12m']
RN50x4 ['openai']
RN50x16 ['openai']
RN50x64 ['openai']
RN101 ['openai', 'yfcc15m']
RN101-quickgelu ['openai', 'yfcc15m']
roberta-ViT-B-32 ['laion2b_s12b_b32k']
swin_base_patch4_window7_224 []
ViT-B-16 ['openai', 'laion400m_e31', 'laion400m_e32', 'laion2b_s34b_b88k', 'datacomp_xl_s13b_b90k', 'datacomp_l_s1b_b8k', 'commonpool_l_clip_s1b_b8k', 'commonpool_l_laion_s1b_b8k', 'commonpool_l_image_s1b_b8k', 'commonpool_l_text_s1b_b8k', 'commonpool_l_basic_s1b_b8k', 'commonpool_l_s1b_b8k']
ViT-B-16-plus []
ViT-B-16-plus-240 ['laion400m_e31', 'laion400m_e32']
ViT-B-32 ['openai', 'laion400m_e31', 'laion400m_e32', 'laion2b_e16', 'laion2b_s34b_b79k', 'datacomp_xl_s13b_b90k', 'datacomp_m_s128m_b4k', 'commonpool_m_clip_s128m_b4k', 'commonpool_m_laion_s128m_b4k', 'commonpool_m_image_s128m_b4k', 'commonpool_m_text_s128m_b4k', 'commonpool_m_basic_s128m_b4k', 'commonpool_m_s128m_b4k', 'datacomp_s_s13m_b4k', 'commonpool_s_clip_s13m_b4k', 'commonpool_s_laion_s13m_b4k', 'commonpool_s_image_s13m_b4k', 'commonpool_s_text_s13m_b4k', 'commonpool_s_basic_s13m_b4k', 'commonpool_s_s13m_b4k']
ViT-B-32-256 ['datacomp_s34b_b86k']
ViT-B-32-plus-256 []
ViT-B-32-quickgelu ['openai', 'laion400m_e31', 'laion400m_e32']
ViT-bigG-14 ['laion2b_s39b_b160k']
ViT-e-14 []
ViT-g-14 ['laion2b_s12b_b42k', 'laion2b_s34b_b88k']
ViT-H-14 ['laion2b_s32b_b79k']
ViT-H-16 []
ViT-L-14 ['openai', 'laion400m_e31', 'laion400m_e32', 'laion2b_s32b_b82k', 'datacomp_xl_s13b_b90k', 'commonpool_xl_clip_s13b_b90k', 'commonpool_xl_laion_s13b_b90k', 'commonpool_xl_s13b_b90k']
ViT-L-14-280 []
ViT-L-14-336 ['openai']
ViT-L-16 []
ViT-L-16-320 []
ViT-M-16 []
ViT-M-16-alt []
ViT-M-32 []
ViT-M-32-alt []
ViT-S-16 []
ViT-S-16-alt []
ViT-S-32 []
ViT-S-32-alt []
vit_medium_patch16_gap_256 []
vit_relpos_medium_patch16_cls_224 []
xlm-roberta-base-ViT-B-32 ['laion5b_s13b_b90k']
xlm-roberta-large-ViT-H-14 ['frozen_laion5b_s13b_b90k']

[vvolhejn]

Hi! If this something that you find useful, we welcome pull requests. Adding more models requires some manual work because we do need to convert the models to ONNX and then host them in our S3 bucket. The conversion part can be done in clip_to_onnx.py, and regarding hosting, you can talk to us and we can host the models. For example, if you extend the clip_to_onnx.py script to export the models that you are interested in.

Additionally, the embedding sizes are different for different models, and there is some simple logic for getting the embedding size that would need to be extended.

[QIN2DIM]

Does it have to be compatible with python 3.6?😮 I've made some changes, but I need to use dataclass.（But dataclass is not necessary. It improves readability, and it's certainly better if it can be used.）

[vvolhejn]

This might change in the future, but our product MLTest maintains 3.6 compatibility at the moment. You can use the backport https://pypi.org/project/dataclasses/ to get dataclasses in Python 3.6, though.

[vvolhejn]

After an internal discussion, we did release version 4.0.0, which bumped the minimum Python version to 3.8, the lowest version that still hasn't reached end-of-life.