Regarding fine tuning for Custom VQA dataset

[manas6266]

Thanks for your great work, I am interested in fine-tuning OFA for Visual Question Answering (VQA) using my custom dataset, which includes image-question-answer pairs. However, my dataset lacks confidence scores for the answers. I would like to understand why confidence scores are needed for OFA fine-tuning and how I can handle this absence in my case. Additionally, I've noticed that even the VQA-v2 dataset does not include confidence scores. During inference, will the answers be generated from fixed vocabulary pickle files only, and if so, what is the reason for not using classification models instead of OFA?

[logicwong]

@manas6266

1. You can set the confidence score of each answer to 1.
2. In the original vqav2 dataset, each sample contains multiple answers. We followed the previous works to set the confidence score for each answer based on its frequency.

[manas6266]

What is the max token size we could give to model?

On Thu, 31 Aug 2023 at 20:05, Wang Peng @.***> wrote:

@manas6266 https://github.com/manas6266

1. You can set the confidence score of each answer to 1.
2. In the original vqav2 dataset, each sample contains multiple answers. We followed the previous works to set the confidence score for each answer based on its frequency.

— Reply to this email directly, view it on GitHub https://github.com/OFA-Sys/OFA/issues/408#issuecomment-1701173057, or unsubscribe https://github.com/notifications/unsubscribe-auth/AOWGQIFX5Q2WDEEZCD2MTF3XYCOMVANCNFSM6AAAAAA2YFOM2I . You are receiving this because you were mentioned.Message ID: @.***>