LAVIS
LAVIS copied to clipboard
salesforce
LAVIS - A One-stop Library for Language-Vision Intelligence
Hello, First of all, awesome work Salesforce team. I am finetuning Blip2FlanXL I noticed in base_task.py you are not dividing loss by accum_grad_iters. https://github.com/salesforce/LAVIS/blob/47e0f3f25ca763975738c7224c8369207812ce6c/lavis/tasks/base_task.py#L214-L230 For FlanXL the default loss reduction...
Thank you for your great work! I also want to ask how can I load the weight of _Q-Former_ weight only? Is the any code for reference? Looking forward to...
Hello I have a few questions: 1. I see both pretrain and caption models have the capabilities to generate() caption, I wonder if there is any different or whether one...
``` image = Image.open(image_path).convert("RGB") File "/home/xxxxx/miniconda3/envs/lavis/lib/python3.8/site-packages/PIL/Image.py", line 3092, in open fp = builtins.open(filename, "rb") FileNotFoundError: [Errno 2] No such file or directory: '/export/share/datasets/vision/visual-genome/image/2387109.jpg' ```
Hey , I was trying to test the model of blip2_opt unfortunately I was having issue in loading the model in jupyter. Below attached is the error I obtained. Was...
I had some trouble reproducing InstructBlip model results on the msvd_qa and msrvtt_qa datasets. Could you please tell me what prompt template and hyperparameters were used for these datasets ?...
Wow! I spent so much time to find the "best" parameter settings for BLIP2 to get the very last drop of a bit "more" and was happy with it. And...
INFO - 2024-04-08 03:20:10,560 - base_model - Missing keys [] INFO - 2024-04-08 03:20:10,560 - base_model - load checkpoint from https://storage.googleapis.com/sfr-vision-language-research/LAVIS/models/BLIP/blip_coco_caption_base.pth
When I execute the following code, I cannot connect. But other models are connectable, what causes this? OSError: We couldn't connect to 'https://huggingface.co' to load this file, couldn't find it...
from lavis.models import load_model_and_preprocess i dont know where the load_model_and_preprocess is，thanks！
Metadata
Owner
Metadata
LAVIS - A One-stop Library for Language-Vision Intelligence