generative-ai-python
generative-ai-python copied to clipboard
google-gemini
I am a paid Gemini Enterprise customer. I STILL get Frequent google.api_core.exceptions.InternalServerError: 500 An internal error has occurred
Description of the bug:
I am a paid Gemini Enterprise customer. I STILL get Frequent google.api_core.exceptions.InternalServerError: 500 An internal error has occurred
THIS IS NOT deterministic. Sometimes things work and sometime sthings do NOT
File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/google/ai/generativelanguage_v1beta/services/generative_service/client.py", line 566, in generate_content response = rpc( ^^^^ File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/google/api_core/gapic_v1/method.py", line 131, in call return wrapped_func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/google/api_core/retry/retry_unary.py", line 293, in retry_wrapped_func return retry_target( ^^^^^^^^^^^^^ File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/google/api_core/retry/retry_unary.py", line 153, in retry_target _retry_error_helper( File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/google/api_core/retry/retry_base.py", line 212, in _retry_error_helper raise final_exc from source_exc File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/google/api_core/retry/retry_unary.py", line 144, in retry_target result = target() ^^^^^^^^ File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/google/api_core/timeout.py", line 120, in func_with_timeout return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/google/api_core/grpc_helpers.py", line 78, in error_remapped_callable raise exceptions.from_grpc_error(exc) from exc google.api_core.exceptions.InternalServerError: 500 An internal error has occurred. Please retry or report in https://developers.generativeai.google/guide/troubleshooting
Actual vs expected behavior:
No response
Any other information you'd like to share?
No response
This is a known issue and they are working on it 500 An internal error is the model overloaded. There are a lot of people using this system due to the fact that its free. this and 503 are the number one errors reported in the Discord channel right now.
Hi, this error occurs when the chat history becomes too long or large. Probably a token issue.
To work around this please prune your chat history periodically as described in the medium post Avoiding InternalServerError When Using Google GenerativeAI Chat.
It took me a while to track this down but it seems to solve the problem until the Gemini team can improve the error codes coming back from the API.
his error occurs when the chat history becomes too long or large. Probably a token issue.
Oh, yes. I've seen that error too, I've raised this internally. You can tell if this is the problem by checking the output of genai.count_tokens and comparing that to the model's maximum input context. 1.5-pro has much longer context than 1.0-pro.
ref: https://ai.google.dev/gemini-api/docs/models/gemini
500 can be a lot of things, overloaded servers, uncaught exceptions, in this case it sounds like @pkenjora pinpointed the actual problem, it was throwing a 500 instead of "invalid argument" when the context gets too long (I've reported this internally). Closing.
Is it common for "I've reported this internally" to be a reason for closing an issue? It seems like you'd want to keep the issue open until it is resolved?
