Google Gemini Vision - Error with Request. (ACCESS_TOKEN_SCOPE_INSUFFICIENT)

[haseeb-heaven]

Request: Open calculator and add 23 + 35

Output:

Error parsing JSON: 403 Request had insufficient authentication scopes. [
<img width="1309" alt="image" src="https://github.com/OthersideAI/self-operating-computer/assets/11544739/671e5ec2-326c-422b-8f0f-d87d58c3bd20">
reason: "ACCESS_TOKEN_SCOPE_INSUFFICIENT"
domain: "googleapis.com"
metadata {
  key: "service"
  value: "generativelanguage.googleapis.com"
}
metadata {
  key: "method"
  value: "google.ai.generativelanguage.v1beta.GenerativeService.GenerateContent"
}
]
[Self-Operating Computer][Error] something went wrong :(
[Self-Operating Computer][Error] AI response
Failed take action after looking at the scr
<img width="1309" alt="image" src="https://github.com/OthersideAI/self-operating-computer/assets/11544739/b9ab8176-860d-4794-8fe3-4b1049658cfb">
eenshot

[joshbickett]

Hi @haseeb-heaven, I had a similar issue. It appears that requesting a key from the Makersuite is not sufficient to use Gemini locally on your laptop.

It requires setting up a Google test project as well by following the directions at this URL: https://ai.google.dev/palm_docs/oauth_quickstart

Objectives

• Set up your cloud project for OAuth
• Set up application-default-credentials
• Manage credentials in your program instead of using gcloud auth

Let me know if this doesn't work for you. The process is more work than I originally realized as well, but I was able to get it to work locally.

If you have ideas on how to improve the README.MD Gemini section, let me know!

[haseeb-heaven]

I have used Google gemini pro and vision pro in my Python projects hosted offline and online they work fine and doesn’t need any Auth setup? So are you sure about this we need this setup .

Check my project https://github.com/haseeb-heaven/Gemini-Vision-Pro it uses Google gemini vision pro and works fine locally and doesn’t need any extra setups.

I think we need more logs to this to track this issue

[joshbickett]

@haseeb-heaven oh ok. It is possible you don't need the credentials then. I wasn't able to get it working without them but I may have been doing something wrong.

[haseeb-heaven]

I will test it again today and will update about this issue will have to check or update the logs then from server

[sgn1989]

I was able to use Gemini-pro without the Google Cloud credentials but while using Gemini-vision-pro, I got an error relating to Google Cloud credentials. So I have set up the google cloud CLI but still, when I try to use the Vision Pro, I am getting this error. I am not sure how to make the credentials accessible in the Streamlit App.

@joshbickett Please let me know if you have any suggestions. Thanks.

[haseeb-heaven]

But we have Google Gemini Vision from Google AI studio not from Google Vertex AI. There is difference Google Vertex AI requires credentials files from JSON while Google Studio AI. requires only single API key.

[sgn1989]

I am not sure, I generated the API key from Google Studio AI only. I tested the Gemini model and it generates the response but when I use the same key for the Vision model, it fails with this error. Not sure where I am going wrong.

[haseeb-heaven]

Same error for Vision its not working well need update

[michsiu]

hello @joshbickett, May I know how to finish the credentials? Would you please provide some tutorials? I think we met the same issue, because it raise an error about network and tcp handshaker shutdown Thanks

[michsiu]

I have another idea about this issue, if it's the reason that the region restrict make the device cannot connect with the Gemini pro? The same api key I tested can work on my phone but cannot work on computer

[haseeb-heaven]

We have to check the logs to get the exact error from project

[AbdulrahmanAbdullahAI]

I have the same issue, I am using Gemini pro vision. it works well locally, but when deployed to streamlit cloud it shows exactly the same error