generative-ai-python icon indicating copy to clipboard operation
generative-ai-python copied to clipboard

Add client timeout setting to overcome timeout errors.

Open homer6 opened this issue 9 months ago • 1 comments

Description of the feature request:

Add client timeout setting to overcome timeout errors.

What problem are you trying to solve with this feature?

I'm encountering this error where I can't configure the client's timeout.

Any other information you'd like to share?

  File "/app/ai.py", line 97, in predict
    response = chat.send_message(
  File "/opt/venv/lib/python3.10/site-packages/google/generativeai/generative_models.py", line 474, in send_message
    response = self.model.generate_content(
  File "/opt/venv/lib/python3.10/site-packages/google/generativeai/generative_models.py", line 262, in generate_content
    response = self._client.generate_content(
  File "/opt/venv/lib/python3.10/site-packages/google/ai/generativelanguage_v1beta/services/generative_service/client.py", line 791, in generate_content
    response = rpc(
  File "/opt/venv/lib/python3.10/site-packages/google/api_core/gapic_v1/method.py", line 131, in __call__
    return wrapped_func(*args, **kwargs)
  File "/opt/venv/lib/python3.10/site-packages/google/api_core/retry/retry_unary.py", line 293, in retry_wrapped_func
    return retry_target(
  File "/opt/venv/lib/python3.10/site-packages/google/api_core/retry/retry_unary.py", line 153, in retry_target
    _retry_error_helper(
  File "/opt/venv/lib/python3.10/site-packages/google/api_core/retry/retry_base.py", line 212, in _retry_error_helper
    raise final_exc from source_exc
  File "/opt/venv/lib/python3.10/site-packages/google/api_core/retry/retry_unary.py", line 144, in retry_target
    result = target()
  File "/opt/venv/lib/python3.10/site-packages/google/api_core/timeout.py", line 120, in func_with_timeout
    return func(*args, **kwargs)
  File "/opt/venv/lib/python3.10/site-packages/google/api_core/grpc_helpers.py", line 78, in error_remapped_callable
    raise exceptions.from_grpc_error(exc) from exc

Exception Type: DeadlineExceeded 
Exception Value: 504 Deadline Exceeded

homer6 avatar May 12 '24 05:05 homer6

response = chat.send_message(

The SDK supports this, but that was never wired into chat

https://github.com/google-gemini/generative-ai-python/pull/204 https://github.com/google-gemini/generative-ai-python/issues/290

That should be an easy fix.

https://github.com/google-gemini/generative-ai-python/pull/341

MarkDaoust avatar May 14 '24 19:05 MarkDaoust

Thank you @markmcd !

homer6 avatar May 24 '24 02:05 homer6

Oh, it looks like helper_types is not included in the types file:

cannot import name 'helper_types' from 'google.generativeai.types' (/opt/venv/lib/python3.10/site-packages/google/generativeai/types/__init__.py)

@markmcd

homer6 avatar May 24 '24 02:05 homer6

It exists in the main branch, but not the v0.5.4 tag.

homer6 avatar May 24 '24 02:05 homer6