Lets get a sample of standard retry logic with exponential backoff, etc.

[SeaDude]

trafficstars

Description of the feature request:

There are many recipes for all sorts of functionality, but none (that I can find) that show retry logic for return codes 429, 503 and 500. I'm seeing these return codes A LOT.

What problem are you trying to solve with this feature?

More robust API calls.

Any other information you'd like to share?

This snippet to successfully retry when return code is 429 Resource Exhausted but times-out if return code is 503 Model is Overloaded or if 500 An internal error has occurred.

from google.generativeai.types import RequestOptions
from google.api_core import retry

def submit_gemini_query(api_key, system_message, user_message, response_class):
    
    genai.configure(api_key=api_key)

    generation_config = {
        "temperature": 0,
        "max_output_tokens": 8192
    }
    
    model = genai.GenerativeModel(
        model_name="gemini-1.5-pro-latest",
        generation_config=generation_config,
        system_instruction=system_message
    )

    response = model.generate_content(user_message,
                                      request_options=RequestOptions(
                                        retry=retry.Retry(
                                            initial=10, 
                                            multiplier=2, 
                                            maximum=60, 
                                            timeout=300
                                        )
                                       )
                                    )

    return response.text

• 
• 
• 

[MarkDaoust]

It's used in this example:

https://github.com/google-gemini/cookbook/blob/7b069178ea852fe6c91c51b576282273dfd04f24/examples/Story_Writing_with_Prompt_Chaining.ipynb#L121

But cookbook could use a walkthrough of the http_options.

[Giom-V]

The issue is that at the moment the new SDK doesn't support retries, we have planned to write a notebook about errors and retries when it will be supported.

[markmcd]

Agree we want this. Here is the google.api_core.retry API reference, in case anyone stumbles on this.

I've been working on this a bit recently so here's how you can use it with the new SDK. It's not ideal, we're working on getting it built in more naturally, so we won't include this in the cookbook unless absolutely necessary (e.g. large embedding batches):

from google.api_core import retry

# Catch transient Gemini errors.
def is_retryable(e) -> bool:
    if retry.if_transient_error(e):
        # Good practice, but probably won't fire with the google-genai SDK
        return True
    elif (isinstance(e, genai.errors.ClientError) and e.code == 429):
        # Catch 429 quota exceeded errors
        return True
    elif (isinstance(e, genai.errors.ServerError) and e.code == 503):
        # Catch 503 model overloaded errors
        return True
    else:
        return False

@retry.Retry(predicate=is_retryable)
def do_stuff(...):
    return client.models.generate_content(...).text

do_stuff(...)

The specific errors will need tweaking but this at least gives a template to start with.

[pcalatayud-prog]

This is exactly what I was looking for! I am facing 503 error when asyncIO calls to gemini