TVLT inaccurate VQA score

inaccurate VQA score

Open Park-ing-lot opened this issue 2 years ago • 3 comments

When I finetune the model with VQAv2, It produces a lower score than that proposed in the paper. (43%)

Can you explain where this difference comes from and how to fix it??

Feb 24 '23 01:02 Park-ing-lot

How many epochs did you run. Did you properly initialized the model weights?

Feb 24 '23 01:02 zinengtang

10 epochs and I initialized with pre-trained TVLT weights from huggingface named TVLT.ckpt

Feb 24 '23 01:02 Park-ing-lot

Hi! I'm experiencing the same issue and have found no performance difference between randomly initialized models and pre-trained checkpoints.

So, one suspicion is there might be some problems in the initialization. Is there a way to check if the model is properly initialized?

Mar 05 '23 09:03 irishev