Where to find the Evaluation Scripts for InstructBLIP paper's Table 1？

[weiyueli7]

Hello,

I wonder where can I find all of the evaluation scripts (or partial) to reproduce Table 1 of the InstructBLIP paper. I tried to reproduce the evaluation results for some of the dataset but it is much worse than what the paper has mentioned. Therefore I would like to see the scripts so that I can figure out what goes wrong.

I am referring to the table here

Thank you!

[SZhanZ]

I also met the same problem. For example, I only achieve the 40.72acc on the GQA test-dev for vicuna7b. And my used checkpoint is just the weight recorded in lavis/config/models/blip2_instruct_vicuna7b.yaml. https://storage.googleapis.com/sfr-vision-language-research/LAVIS/models/InstructBLIP/instruct_blip_vicuna7b_trimmed.pth

Do you address it?

[Fym68]

Could you provide evaluation script or implementation guideline for okvqa？Sincere thanks！ @SZhanZ @weiyueli7