aokvqa icon indicating copy to clipboard operation
aokvqa copied to clipboard

there is different performance from the paper

Open huangsiyong opened this issue 2 years ago • 1 comments

Hi, it is an interesting work. However, I find something different from the your paper. I try to evaluate the model loaded the provided pretrained weight. I find the DA acc higher a little than the paper show one but all the MC acc is one point lower. What's wrong? Is it any change about the dataset?

huangsiyong avatar Feb 22 '23 07:02 huangsiyong

Hi @huangsiyong, thanks for bringing this up. We did actually make some small changes to the dataset between our last post to arxiv and publication. Apologies for the delay in revision, but we will update the arxiv release very soon.

apoorvkh avatar Feb 27 '23 17:02 apoorvkh

I think we can defer to the published ECCV paper, with updated numbers, instead:

https://www.ecva.net/papers/eccv_2022/papers_ECCV/papers/136680141.pdf

And I've updated the link in this repo's readme.

apoorvkh avatar May 08 '24 16:05 apoorvkh