load model "blip2_image_text_matching" failed

[dragonwhl]

Tested blip2_image_text_matching.ipynb in Colab,run at model, vis_processors, text_processors = load_model_and_preprocess("blip2_image_text_matching", "pretrain", device=device, is_eval=True) got "AttributeError: 'NoneType' object has no attribute 'from_pretrained'" . And Try : from lavis.models import model_zoo print(model_zoo)

found no Architecture "blip2_image_text_matching".

[dxli94]

Hi @dragonwhl, if you install from PyPI, the ITM model is not available yet. If you'd like to use it for now, please try install from the source.

We are adding a few other models, and will publish a new release when they are ready.

[zhzxlcc]

i come to this problem while loading this model: model, vis_processors, text_processors = load_model_and_preprocess("blip2_image_text_matching", "pretrain", device=device, is_eval=True)

File "/home/user/anaconda3/envs/lavis/lib/python3.8/site-packages/transformers/tokenization_utils_base.py", line 1825, in from_pretrained return cls._from_pretrained( File "/home/user/anaconda3/envs/lavis/lib/python3.8/site-packages/transformers/tokenization_utils_base.py", line 1988, in _from_pretrained tokenizer = cls(*init_inputs, **init_kwargs) File "/home/user/anaconda3/envs/lavis/lib/python3.8/site-packages/transformers/models/bert/tokenization_bert.py", line 213, in init if not os.path.isfile(vocab_file): File "/home/user/anaconda3/envs/lavis/lib/python3.8/genericpath.py", line 30, in isfile st = os.stat(path) TypeError: stat: path should be string, bytes, os.PathLike or integer, not NoneType

[rookiiiiiie]

Hi @zhzxlcc I also had the same problem, did you solve it?

[zhzxlcc]

I download the bert_config manually and set the filepath in LAVIS/lavis/models/blip2_models/blip2.py.

tokenizer = BertTokenizer.from_pretrained(pretrained_dir, truncation_side=truncation_side)

I can run blip2_image_text_matching.ipynb now. If you still have the problem, I'll check the changes carefully.