Got the AssertionError message.

[iceleaf97tech]

AssertionError: BLIP models are not compatible with transformers>=4.27, run pip install transformers==4.25 to downgrade

GPU: RTX A6000 CUDA version: 11.7 conda version: 23.3.1

[Reex11]

Same issue

[LiJunnan1992]

Please downgrade transformers if you use BLIP models.

[iceleaf97tech]

When the version of transformer is downgradeed, but this message appears:

pip install transformers==4.25

======================[ERROR message]====================== ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. salesforce-lavis 1.0.1 requires transformers>=4.28.0, but you have transformers 4.25.0 which is incompatible. ========================================================

However, the following code does work.

import torch
from PIL import Image
from lavis.models import load_model_and_preprocess
# setup device to use
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
# load sample image
raw_image = Image.open("library/LAVIS/docs/_static/merlion.png").convert("RGB")

# loads BLIP caption base model, with finetuned checkpoints on MSCOCO captioning dataset.
# this also loads the associated image processors
model, vis_processors, _ = load_model_and_preprocess(name="blip_caption", model_type="base_coco", is_eval=True, device=device)

# preprocess the image
# vis_processors stores image transforms for "train" and "eval" (validation / testing / inference)
image = vis_processors["eval"](raw_image).unsqueeze(0).to(device)
# generate caption
print(model.generate({"image": image}))
# ['a large fountain spewing water into the air']

Output

['a large fountain spewing water into the air']

[chengyangfeng]

when i'am run bash run_scripts/run_demo.sh, the error is same with you,how do you fix it?

[iceleaf97tech]

@chengyangfeng I ran my code mentioned above with python rather than run_demo.sh.

[joaquincabezas]

BLIP is still interesting, so I would suggest including this issue in the repo. Some options are:

• Downgrade transformers in the requirements.txt (I just tested with BLIP2 and it works, don't know about the rest)
• Include a separate requirerements-BLIP.txt (in the root folder or create a requirements folder)
• Include !pip install transformers==4.25 in the BLIP example notebooks and duplicate the demo script (one will install transformers 4.25 in the current environment)