BLIP2 VQA finetune

[evelinehong]

trafficstars

Hi we are working on finetuning VQA with BLIP2. Any instructions on how to modify the codes? When will the finetuning codes be released?

[xliucs]

Same question. I am wondering if there is any updates on this. Thanks!

[fmdmm]

The question is similar： https://github.com/salesforce/LAVIS/issues/125

[qwqwq1445]

Excuse me, I am also working on finetuning VQA on BLIP2. In the paper, I find that the Prompt used for VQA is "Question: {} Answer:". I would like to ask if my understanding is correct: when training, we don't utilize the prompt and only use the original question input; when testing, we utilize the prompt to reformat the question input to get a better performance. I will appreciate it if you could kindly help. Thanks.

[Hurwitzzz]

Excuse me, I am also working on finetuning VQA on BLIP2. In the paper, I find that the Prompt used for VQA is "Question: {} Answer:". I would like to ask if my understanding is correct: when training, we don't utilize the prompt and only use the original question input; when testing, we utilize the prompt to reformat the question input to get a better performance. I will appreciate it if you could kindly help. Thanks.

Hi, I have the same question. According to the forword() function in blip2_t5.py, it seems like prompts are not used during training. But I'm wondering that shouldn't we use the same format during training and evaluating? Did you figure it out? Thanks!