Add support for Salesforce BLIP in image-to-text

[learyjk]

Model description

I’m new to this. Had fun experimenting with image-to-text today but found the gpt2 model was not great. Can we add support for the Salesforce BLIP model somehow?

I’ll look into exporting it to ONNX myself but I didn’t see it on the list for the second checkbox under prerequisites 🙃.

If I’m off base - any recommendations for how to improve the image-to-text pipeline?

Prerequisites

• [X] The model is supported in Transformers (i.e., listed here)
• [ ] The model can be exported to ONNX with Optimum (i.e., listed here)

Additional information

No response

Your contribution

I can submit a PR but I might need a little guidance on the model training piece!

thank you!

[jasonmayes]

+1! I assumed it was available as listed in tasks but it seems to leave error in console when trying to use: Error: Unsupported model type: blip