LAVIS icon indicating copy to clipboard operation
LAVIS copied to clipboard

Published 20 hours ago • verified publisher icon salesforce

zero-shot accuracy of instructblip on ScienceQA

Open tgyy1995 opened this issue 1 year ago • 2 comments

Thank you for your outstanding work.

  1. I noticed that the paper mentions "For ScienceQA, we only evaluate the set with image context." Does this mean that the hint or context in the dataset is not used?
  2. What is the prompt template used by instructblip_flant5xl on ScienceQA?
  3. I use "Question: {}\n{}" as prompt and the zero-shot accuracy of instruct_flant5xl on ScienceQA i obtained is 64.6。Are there any things I need to pay attention to?

tgyy1995 avatar May 24 '23 16:05 tgyy1995