Please give Github Copilot chat VISION 👀

[Khalizo]

Please can you give GitHub copilot eyes using OpenAI's GPT4 vision API?

As you can imagine, the use cases are endless and not limited to:

• Referencing matplotlib charts to help recreate new charts
• Analysing screenshots of websites or apps to create code to recreate those screenshots
• Provided suggestions via code to help improve designs
• etc ...

Here is the link to the API: https://platform.openai.com/docs/guides/vision

Thanks, Babs

[dbsxdbsx]

Yes, I want to pop up this feature. With an image uploaded to Copilot, we can do someting like , for example:

1. For AI code: "Look at the red boxed part in the image, could you tell me how to recog this kind of stuff using OpenCV in python/C++"?
2. For UI design："I want to make a nice looking tab bar just like what I show to you, how to do it using flutter(it is ok if need 3rd party package, just tell me what and how to do)"

[hidaris]

It looks like the cursor has already done this https://twitter.com/amanrsanger/status/1729943964000883186 , hoping to see similar functionality in Copilot

[IAMIbrahimmemon]

what woould be awesome is if you could connect with figma and it would make the UI layout