Google Gemini & Vertex / model version supoprt / API key and IAM support

[zeroasterisk]

I'm collecting a few similar recent feature requests, but this intends to be more broad in scope:

Related FRs:

• [Feature] Support for Google's Gemini! #1176
• Add Google Gemini Pro API #1146
• Gemini Model Errors Persist Despite Overridden Base URL for OpenAI API #1615

Goals:

1. Support for multiple versions of gemini (potentially with experimental release pinning)
2. Support for Vertex AI hosted Model as a Service (including gemini, claude/sonnet, llama, etc [model garden])
3. Support for "host your own model" on Vertex or wherever

Rationale:

• models release quickly, and it'd be nice to have an easy configuration across them
• some companies have requirements about data residency and enterprise access controls and dislike API_KEYs
• some users want to host their own models

Analysis

• [x] 1. Support for multiple versions of gemini (potentially with experimental release pinning)
  • This seems to be functional, I can enter gemini-1.5-flash-exp-0827 which is currently experimental https://ai.google.dev/gemini-api/docs/models/experimental-models
  • It could be simpler to find, or describe in the UI... a lot of guesswork about typing in the model name
• [ ] 2. Support for Vertex AI hosted Model as a Service (including gemini, claude/sonnet, llama, etc model garden)
  • This is not possible today, though you have something similar for Azure.
  • Ask: implement AIM controls and access to general purpose and code completion specific models
    • https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models
    • https://cloud.google.com/vertex-ai/generative-ai/docs/model-garden/explore-models
  • Pro tips:
    • a few models are exposed with the OpenAI API signature https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/call-vertex-using-openai-library
    • some of the models support LCW and caching which is particularly useful for code https://cloud.google.com/vertex-ai/generative-ai/docs/context-cache/context-cache-overview
• [ ] 3. Support for "host your own model" on Vertex or wherever
  • Example https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/community/model_garden/model_garden_pytorch_mixtral_deployment.ipynb
  • Each deployed model is it's own endpoint

Recommendation:

• [ ] [XS] add hints to the UI to configure new models: just type in the full model name and we will figure out how to query it (eg: gemini-1.5-flash-exp-0827)
• [ ] [XS] offer more model versions for selection, perhaps hidden under an accordion or something
• [ ] [L] add support for Vertex AI Model as a Service hosted models and AIM access
• [ ] [L] add support for "bring your own hosted model endpoints" like those exposed by Vertex Model Garden

[yangcheng]

it would be very nice to add vertex claude support!

[XelaNull]

Please consider this soon. I work for a large company that is very interested but won't consider Cursor until this is implemented, to keep it within an enterprise walled garden (that is Anthropic backed). It's painful using Cursor in my off-hours and then using VS Code at work left to copy/paste everything.

[daaain]

I guess an incremental step could be integrating Vertex AI API but leaving AIM to the user, so just instructing people to use ADC – ie log in via gcloud or have a service account JSON with GOOGLE_APPLICATION_CREDENTIALS env var pointing to its path

[shohei-ihaya]

any updates?

[TarekAloui]

can we please add this feature?

[licon]

Our team is in great need of this integration of Google Vetex AI.

[kouji0705]

I also wish for this feature.

[A3Ackerman]

One more request - seems like a huge gap/oversight when Cursor already has bedrock and azure OpenAi integrations etc.

[michael-ravelhq]

Would help us as well