[New task] Visual Question Answering

[xenova]

Feature request

Currently, the visual-question-answering pipeline/task in transformers is not supported for onnx export:

https://github.com/huggingface/optimum/blob/618a483d463b9c2e0b0ba3b859ff74c6279b5161/optimum/exporters/tasks.py#L147-L170

Here are some models which can be used for testing: https://huggingface.co/models?other=visual-question-answering

Motivation

This was brought up as a request for Transformers.js, but I can't do any testing yet until it's supported by optimum :)

Your contribution

I'll implement it in Transformers.js once supported in optimum 👍

[fxmarty]

working on it

[bytesandwich]

Hi @fxmarty that's awesome! Do you have any thoughts on whether this visual-question-answering task for export is viable? I'd like to export something like dandelin/vilt-b32-finetuned-vqa or Salesforce/blip-vqa-base and they both have this type.