Add support for Gemini's Multimodal Live API
Description
https://ai.google.dev/api/multimodal-live
We should add support like we have for OpenAI, and in the process we should unify the interfaces to provide a provider-agnostic Realtime interface.
Somewhat related so I want to add that remark here: Google also introduced a new python SDK --> https://github.com/googleapis/python-genai
This unifies the interface between AI Studio & Vertex AI. So just for that it is worth considering IMO :)
Somewhat related so I want to add that remark here: Google also introduced a new python SDK --> https://github.com/googleapis/python-genai
This unifies the interface between AI Studio & Vertex AI. So just for that it is worth considering IMO :)
I've created a new issue #750 to track this
Closing this in favor of an upcoming RFC on a Realtime API that will support both OpenAI and Google in a provider-agnostic way.