VaLM icon indicating copy to clipboard operation
VaLM copied to clipboard

Published 20 hours ago • verified publisher icon Victorwz

Evaluation prompts

Open paulaonta opened this issue 1 year ago • 1 comments
trafficstars

Hello! I'm trying to replicate your model, and while evaluating the model, I noticed that the prompts don't match those in the paper. Are these the final scripts? Thank you

paulaonta avatar Feb 19 '24 10:02 paulaonta

Hi, one of the 9 prompts for color reasoning in code is different from the paper appendix. I will update the paper accordingly. Please use the prompts in evaluation_scripts/verify_color_prediction.py

Victorwz avatar Feb 21 '24 00:02 Victorwz