Yonatan Bitton

Sure, We follow exact Winoground's evaluation, you can see a very similar code here: https://github.com/yonatanbitton/CLIPEvaluation/blob/main/src/eval_winoground.py The only difference is that instead of CLIP score, we used [BLIP2](https://github.com/salesforce/LAVIS/blob/main/examples/blip2_image_text_matching.ipynb) image-text score. Please...

Hi, thanks for the interest, we are working on the data-release + a public version for the end-to-end model. It will take time, and we will update. Thanks.

Hi, the dataset is updated now in the project website, sorry for the delay, please ask if anything is unclear CSV: https://seetrue.s3.amazonaws.com/wysiwyr_train.csv Images: https://drive.google.com/file/d/1M1CKmYkIdpFYjCOc9JwXHP5Z7E91CJl3/view?usp=drive_link

Thanks for your interest. Sadly we won't be able to share the PaLI checkpoint as this is internal model. However, we do plan to publish the training dataset soon, and...

Hi, Hi, the dataset is updated in the project website, sorry for the delay, please ask if anything is unclear CSV: https://seetrue.s3.amazonaws.com/wysiwyr_train.csv Images: https://drive.google.com/file/d/1M1CKmYkIdpFYjCOc9JwXHP5Z7E91CJl3/view?usp=drive_link

Thanks for your interest. We do plan to publish the training dataset soon, and this way you can reproduce the VNLI training with BLIP2 or any more updated VLM available....

Thanks. Yes we are working on approval for the full data release

Thanks. Yes we are working on approval for the full data release

Hi, it's updated now in the project website, sorry for the delay, please ask if anything is unclear CSV: https://seetrue.s3.amazonaws.com/wysiwyr_train.csv Images: https://drive.google.com/file/d/1M1CKmYkIdpFYjCOc9JwXHP5Z7E91CJl3/view?usp=drive_link

Hi, it's updated now in the project website, sorry for the delay, please ask if anything is unclear CSV: https://seetrue.s3.amazonaws.com/wysiwyr_train.csv Images: https://drive.google.com/file/d/1M1CKmYkIdpFYjCOc9JwXHP5Z7E91CJl3/view?usp=drive_link