Low test accuracy on Refcocog

[shizhediao]

Dear authors, Thanks for your great work! I fine-tuned a VL-T5 model on Refcocog, based on your released checkpoint but got a bad result: 20.45407 I noticed that in your paper the VL-BART diverges but not VL-T5. Could you help take a look?

Thanks!

[TaoHeyi]

Dear authors, Thanks for your great work! I fine-tuned a VL-T5 model on Refcocog, based on your released checkpoint but got a bad result: 20.45407 I noticed that in your paper the VL-BART diverges but not VL-T5. Could you help take a look?

Thanks!

Do you mind if you could share how do you inference on the model of the refcocog? Thanks

[j-min]

Hi, in case you used args.vis_pointer=True in your experiment, would you please try with args.vis_pointer=False, that I made default recently here? I could reproduce the results in the paper with the current codebase.