obsidian-copilot icon indicating copy to clipboard operation
obsidian-copilot copied to clipboard
verified publisher icon logancyang

Using Gemini provided Gemma 3 does not work due to missing parameters on GCP endpoint

Open brando-bang opened this issue 5 months ago • 4 comments

  • [x] Disable all other plugins besides Copilot (required)
  • [x] Screenshot of note + Copilot chat pane + dev console added (required)

Copilot version: v2.9.3

Describe how to reproduce Use an api key for Gemini (GCP) to set up a model that uses one of the Gemma 3 distributions, then start a chat with that model

Expected behavior This should work similar to using the Gemma 3 model from different providers or Gemini 2.5 flavors from GCP

Screenshots Image

Additional context The same issue with developer instructions has appeared in other projects, like this issue in the vercel repo

Seems like the developer and system instruction options are not available in Google's API for gemma 3 models. OpenRouter's downstream providers may have additional configuration on their APIs or they may handle it similarly to how Vercel does. This solution, moving the config params to be pretext in the main prompt body, is much more feasible for Copilot as ticketing Google to change the Gemma api will probably be fruitless

Allowing for the Gemini API's Gemma 3 models is desirable because Gemma 3 has much higher rate limits than Gemini does in the GCP free tier and the 27b Gemma 3 is fairly powerful especially for the size.

brando-bang avatar Jul 06 '25 21:07 brando-bang

Have you tried adding it with "openai format" instead of gemini? Let me know if that works. There are too many variants of google genai clients it's impossible to keep up...

logancyang avatar Jul 08 '25 18:07 logancyang

I wasn't able to get that to work but yeah it's pretty unwiedly that they have so many

brando-bang avatar Jul 08 '25 20:07 brando-bang

For personal use, I could make the gemma variants work by replacing few lines in the Hie of main.js. What I did was simply to branch out the processing by the selected model name, and if the selected model name includes 'gemma', it skips adding system role elements into the transaction, and instead, add them into user prompt. I don't know if I am allowed to share those codes here, so I'm just sharing the brief of it.

Argoneon1810 avatar Jul 22 '25 12:07 Argoneon1810

@Argoneon1810 that's great! You should consider opening a PR in this repo with your fix

brando-bang avatar Aug 02 '25 00:08 brando-bang