aokvqa
aokvqa copied to clipboard
there is different performance from the paper
Hi, it is an interesting work. However, I find something different from the your paper. I try to evaluate the model loaded the provided pretrained weight. I find the DA acc higher a little than the paper show one but all the MC acc is one point lower. What's wrong? Is it any change about the dataset?
Hi @huangsiyong, thanks for bringing this up. We did actually make some small changes to the dataset between our last post to arxiv and publication. Apologies for the delay in revision, but we will update the arxiv release very soon.
I think we can defer to the published ECCV paper, with updated numbers, instead:
https://www.ecva.net/papers/eccv_2022/papers_ECCV/papers/136680141.pdf
And I've updated the link in this repo's readme.