Video Question Answering examples

[behroozazarkhalili]

Hi there, Could you please provide an example as how we should run Video Question Answering task using LAVIS?

Any examples about other video-related tasks well be very appreciated.

[dxli94]

Hi @behroozazarkhalili,

I will work on this, aiming to get back in 1-2 weeks.

Thanks.

[behroozazarkhalili]

Hi @behroozazarkhalili,

I will work on this, aiming to get back in 1-2 weeks.

Thanks.

Thank you for your time and consideration.

[behroozazarkhalili]

@dxli94 Hi. Any update about this?

[behroozazarkhalili]

@mgomes @mgomes @amerine @dxli94 Hi everyone. Could you please kindly do a favor and answer this vital question?

[tobyperrett]

Hi. Will some examples be provided? Thanks.

[dxli94]

Haven't got time to work on this yet.

With the unified interface in LAVIS to load model and preprocess samples, a working example will look similar to others in vqa etc.

PR welcomed.

[tobyperrett]

It looks like the ALPRO model does not have a predict_answers method. Would this would be needed to be implemented in the ALPRO class to do video QA as in the example for vqa using BLIP? Or am I missing something?

[qq846511277]

May I ask how we can use LAVIS to run video Q&A tasks as an example? Is there any latest progress

[ee2110]

Hi, I am also keen to know how to use InstructBLIP for video input, does anyone willing to share? I truly appreciate your valuable time.