google-gemini
Is upload_file thread safe?
Description of the bug:
when I want use genai , I need to use this method in an interface request, but when I put "genai.upload_file(path=path,display_name=file)" into the interface method, it will report an error SLL_ERROR
@app.route("/xxx",method=["GET"]) def upload_video(): output_frame_folder = requests.output_frame_folder time.sleep(1.0) lists = [] try: for file in os.listdir(f"{output_frame_folder}"): path = os.path.join(f"{output_frame_folder}", file) sample_file = genai.upload_file(path=path,display_name=file)
Actual vs expected behavior:
No response
Any other information you'd like to share?
No response
the error desc:[SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:2621)
@jiupinjiandingshi, I tried running Prompting with media files tutorial and genai.upload_file works without any issues. Can you please share a colab gist or code which we can try to reproduce on our end. Thank you!
If I run it once, there is no problem, as follows：
for file in os.listdir(f"{output_frame_folder}"): path = os.path.join(f"{output_frame_folder}", file) sample_file = genai.upload_file(path=path,display_name=file)
But if my interface receives two requests at the same time, this code may be called twice in the same time period. In this case, an error will be reported as follows:
httplib2.Http() error:[SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:2621)
pit:’output_frame_folder ‘ It is a folder that contains some pictures and an MP3
try to run pip install google-generativeai --upgrade that solved it for me!
i was using google-generativeai-0.3.2 but now its google-ai-generativelanguage-0.6.4
Thanks, please reopen this if the problem comes back in a reproducible way.
i tried upgrading the library as suggested by @sMx7d . But still getting errors
File "/code/vision_models.py", line 228, in media2concept
video-backend | messages = [
video-backend | ^
video-backend | File "/code/vision_models.py", line 231, in <listcomp>
video-backend | "parts": await asyncio.create_task(make_video_prompt(media))
video-backend | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
video-backend | File "/code/vision_models.py", line 154, in make_video_prompt
video-backend | framefiles = await upload_frames_concurrently(framefiles)
video-backend | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
video-backend | File "/code/vision_models.py", line 136, in upload_frames_concurrently
video-backend | uploaded_frames = await asyncio.gather(*tasks)
video-backend | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
video-backend | File "/code/vision_models.py", line 127, in aupload_frame
video-backend | res = await asyncio.to_thread(genai.upload_file, ff.file_path, name=ff.name)
video-backend | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
video-backend | File "/usr/local/lib/python3.11/asyncio/threads.py", line 25, in to_thread
video-backend | return await loop.run_in_executor(None, func_call)
video-backend | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
video-backend | File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 58, in run
video-backend | result = self.fn(*self.args, **self.kwargs)
video-backend | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
video-backend | File "/usr/local/lib/python3.11/site-packages/google/generativeai/files.py", line 69, in upload_file
video-backend | response = client.create_file(
video-backend | ^^^^^^^^^^^^^^^^^^^
video-backend | File "/usr/local/lib/python3.11/site-packages/google/generativeai/client.py", line 79, in create_file
video-backend | return self.get_file({"name": result["file"]["name"]})
video-backend | ~~~~~~^^^^^^^^
video-backend | TypeError: string indices must be integers, not 'str'
anyone else finding the same issue?
@sMx7d somehow its gone back to giving the ssl error
these are the installed libraries: google-ai-generativelanguage==0.6.4 google-api-core==2.19.0 google-api-python-client==2.130.0 google-auth==2.29.0 google-auth-httplib2==0.2.0 google-generativeai==0.5.4 googleapis-common-protos==1.63.0 grpcio==1.64.0 grpcio-status==1.62.2 h11==0.14.0 httpcore==1.0.5 httplib2==0.22.0 httpx==0.27.0
This is the same old error trace im getting:
video-backend | uploaded_frames = await asyncio.gather(*tasks)
video-backend | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
video-backend | File "/code/vision_models.py", line 127, in aupload_frame
video-backend | res = await asyncio.to_thread(genai.upload_file, ff.file_path, name=ff.name)
video-backend | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
video-backend | File "/usr/local/lib/python3.11/asyncio/threads.py", line 25, in to_thread
video-backend | return await loop.run_in_executor(None, func_call)
video-backend | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
video-backend | File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 58, in run
video-backend | result = self.fn(*self.args, **self.kwargs)
video-backend | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
video-backend | File "/usr/local/lib/python3.11/site-packages/google/generativeai/files.py", line 69, in upload_file
video-backend | response = client.create_file(
video-backend | ^^^^^^^^^^^^^^^^^^^
video-backend | File "/usr/local/lib/python3.11/site-packages/google/generativeai/client.py", line 77, in create_file
video-backend | result = request.execute()
video-backend | ^^^^^^^^^^^^^^^^^
video-backend | File "/usr/local/lib/python3.11/site-packages/googleapiclient/_helpers.py", line 130, in positional_wrapper
video-backend | return wrapped(*args, **kwargs)
video-backend | ^^^^^^^^^^^^^^^^^^^^^^^^
video-backend | File "/usr/local/lib/python3.11/site-packages/googleapiclient/http.py", line 902, in execute
video-backend | _, body = self.next_chunk(http=http, num_retries=num_retries)
video-backend | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
video-backend | File "/usr/local/lib/python3.11/site-packages/googleapiclient/_helpers.py", line 130, in positional_wrapper
video-backend | return wrapped(*args, **kwargs)
video-backend | ^^^^^^^^^^^^^^^^^^^^^^^^
video-backend | File "/usr/local/lib/python3.11/site-packages/googleapiclient/http.py", line 1007, in next_chunk
video-backend | resp, content = _retry_request(
video-backend | ^^^^^^^^^^^^^^^
video-backend | File "/usr/local/lib/python3.11/site-packages/googleapiclient/http.py", line 222, in _retry_request
video-backend | raise exception
video-backend | File "/usr/local/lib/python3.11/site-packages/googleapiclient/http.py", line 191, in _retry_request
video-backend | resp, content = http.request(uri, method, *args, **kwargs)
video-backend | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
video-backend | File "/usr/local/lib/python3.11/site-packages/httplib2/__init__.py", line 1724, in request
video-backend | (response, content) = self._request(
video-backend | ^^^^^^^^^^^^^^
video-backend | File "/usr/local/lib/python3.11/site-packages/httplib2/__init__.py", line 1444, in _request
video-backend | (response, content) = self._conn_request(conn, request_uri, method, body, headers)
video-backend | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
video-backend | File "/usr/local/lib/python3.11/site-packages/httplib2/__init__.py", line 1396, in _conn_request
video-backend | response = conn.getresponse()
video-backend | ^^^^^^^^^^^^^^^^^^
video-backend | File "/usr/local/lib/python3.11/http/client.py", line 1395, in getresponse
video-backend | response.begin()
video-backend | File "/usr/local/lib/python3.11/http/client.py", line 325, in begin
video-backend | version, status, reason = self._read_status()
video-backend | ^^^^^^^^^^^^^^^^^^^
video-backend | File "/usr/local/lib/python3.11/http/client.py", line 286, in _read_status
video-backend | line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
video-backend | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
video-backend | File "/usr/local/lib/python3.11/socket.py", line 706, in readinto
video-backend | return self._sock.recv_into(b)
video-backend | ^^^^^^^^^^^^^^^^^^^^^^^
video-backend | File "/usr/local/lib/python3.11/ssl.py", line 1314, in recv_into
video-backend | return self.read(nbytes, buffer)
video-backend | ^^^^^^^^^^^^^^^^^^^^^^^^^
video-backend | File "/usr/local/lib/python3.11/ssl.py", line 1166, in read
video-backend | return self._sslobj.read(len, buffer)
video-backend | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
video-backend | ssl.SSLError: [SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:2580)
Its stil a ssl error , sorry but i don't know anything in ssl or what it means . But it seems that genertiveai version is 0.5.4 , IDK if it has upload file function or not , but either way, i don't think upgrading the library will solve as its a ssl error and not related to library functions. I hope my response was clear, if you needed anything just mention me. @ronfromhp
@jiupinjiandingshi can you reopen the issue? As currently I havent found a resolution which works
video-backend | TypeError: string indices must be integers, not 'str'
sorry , i just reviewed the error and i saw video-backend | TypeError: string indices must be integers, not 'str'
that error is likely not from the generative-ai library , its from your code but IDK what the solve for it.
as of my previous comment i was still getting the same ssl error even after updating genai @sMx7d
Yeah, this error is reproducible, sorry I missed the follow ups here after the initial close.
This script works fine:
https://gist.github.com/MarkDaoust/dcd65b626bf4683860aa510b79bc225e
But if I add an upload_file to the code that runs in the threads I get a messy SSL error:
https://gist.github.com/MarkDaoust/73035a173d532f03c1a2c5aa49de9993
Traceback (most recent call last):
File "/Users/markdaoust/Projects/generative-ai-python/run_upload_threads.py", line 164, in <module>
for n, (prompt, response) in enumerate(zip(prompts, responses)):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/markdaoust/homebrew/Cellar/[email protected]/3.12.5/Frameworks/Python.framework/Versions/3.12/lib/python3.12/concurrent/futures/_base.py", line 619, in result_iterator
yield _result_or_cancel(fs.pop())
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/markdaoust/homebrew/Cellar/[email protected]/3.12.5/Frameworks/Python.framework/Versions/3.12/lib/python3.12/concurrent/futures/_base.py", line 317, in _result_or_cancel
return fut.result(timeout)
^^^^^^^^^^^^^^^^^^^
File "/Users/markdaoust/homebrew/Cellar/[email protected]/3.12.5/Frameworks/Python.framework/Versions/3.12/lib/python3.12/concurrent/futures/_base.py", line 456, in result
return self.__get_result()
^^^^^^^^^^^^^^^^^^^
File "/Users/markdaoust/homebrew/Cellar/[email protected]/3.12.5/Frameworks/Python.framework/Versions/3.12/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
raise self._exception
File "/Users/markdaoust/homebrew/Cellar/[email protected]/3.12.5/Frameworks/Python.framework/Versions/3.12/lib/python3.12/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/markdaoust/Projects/generative-ai-python/run_upload_threads.py", line 157, in get_result
f = genai.upload_file(fpath)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/markdaoust/Projects/generative-ai-python/google/generativeai/files.py", line 85, in upload_file
response = client.create_file(
^^^^^^^^^^^^^^^^^^^
File "/Users/markdaoust/Projects/generative-ai-python/google/generativeai/client.py", line 121, in create_file
result = request.execute()
^^^^^^^^^^^^^^^^^
File "/Users/markdaoust/Projects/venv3/lib/python3.12/site-packages/googleapiclient/_helpers.py", line 130, in positional_wrapper
return wrapped(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/markdaoust/Projects/venv3/lib/python3.12/site-packages/googleapiclient/http.py", line 902, in execute
_, body = self.next_chunk(http=http, num_retries=num_retries)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/markdaoust/Projects/venv3/lib/python3.12/site-packages/googleapiclient/_helpers.py", line 130, in positional_wrapper
return wrapped(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/markdaoust/Projects/venv3/lib/python3.12/site-packages/googleapiclient/http.py", line 1084, in next_chunk
resp, content = http.request(
^^^^^^^^^^^^^
File "/Users/markdaoust/Projects/venv3/lib/python3.12/site-packages/httplib2/__init__.py", line 1724, in request
(response, content) = self._request(
^^^^^^^^^^^^^^
File "/Users/markdaoust/Projects/venv3/lib/python3.12/site-packages/httplib2/__init__.py", line 1444, in _request
(response, content) = self._conn_request(conn, request_uri, method, body, headers)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/markdaoust/Projects/venv3/lib/python3.12/site-packages/httplib2/__init__.py", line 1396, in _conn_request
response = conn.getresponse()
^^^^^^^^^^^^^^^^^^
File "/Users/markdaoust/homebrew/Cellar/[email protected]/3.12.5/Frameworks/Python.framework/Versions/3.12/lib/python3.12/http/client.py", line 1428, in getresponse
response.begin()
File "/Users/markdaoust/homebrew/Cellar/[email protected]/3.12.5/Frameworks/Python.framework/Versions/3.12/lib/python3.12/http/client.py", line 331, in begin
version, status, reason = self._read_status()
^^^^^^^^^^^^^^^^^^^
File "/Users/markdaoust/homebrew/Cellar/[email protected]/3.12.5/Frameworks/Python.framework/Versions/3.12/lib/python3.12/http/client.py", line 292, in _read_status
line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/markdaoust/homebrew/Cellar/[email protected]/3.12.5/Frameworks/Python.framework/Versions/3.12/lib/python3.12/socket.py", line 720, in readinto
return self._sock.recv_into(b)
^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/markdaoust/homebrew/Cellar/[email protected]/3.12.5/Frameworks/Python.framework/Versions/3.12/lib/python3.12/ssl.py", line 1252, in recv_into
return self.read(nbytes, buffer)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/markdaoust/homebrew/Cellar/[email protected]/3.12.5/Frameworks/Python.framework/Versions/3.12/lib/python3.12/ssl.py", line 1104, in read
return self._sslobj.read(len, buffer)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ssl.SSLError: [SSL] record layer failure (_ssl.c:2559)
Upload file is one method that isn't auto-generated in the google.ai.generativelanguage client library.
I added code client.py#L69-L123 to implement the API's create_file, since the generated client doesn't handle it.
This code is based on the discovery API, and I shouldn't be sharing those discovery instances between threads:
- https://googleapis.github.io/google-api-python-client/docs/thread_safety.html
- https://github.com/doitintl/doit-easily-marketplace/issues/54
Thanks @jachor for pointing this out.
So I think we just need to make discovery_api thread local here:
https://github.com/google-gemini/generative-ai-python/blob/main/google/generativeai/client.py#L86-L88
Also related: https://github.com/google-gemini/generative-ai-python/issues/564
class FileServiceClient(glm.FileServiceClient):
def __init__(self, *args, **kwargs):
self._local = threading.local()
super().__init__(*args, **kwargs)
def _setup_discovery_api(self, metadata: dict | Sequence[tuple[str, str]] = ()):
if not hasattr(self._local, 'discovery_api'):
api_key = self._client_options.api_key
if api_key is None:
raise ValueError(
"Invalid operation: Uploading to the File API requires an API key. Please provide a valid API key."
)
request = googleapiclient.http.HttpRequest(
http=httplib2.Http(),
postproc=lambda resp, content: (resp, content),
uri=f"{GENAI_API_DISCOVERY_URL}?version=v1beta&key={api_key}",
headers=dict(metadata),
)
response, content = request.execute()
request.http.close()
discovery_doc = content.decode("utf-8")
self._local.discovery_api = googleapiclient.discovery.build_from_document(
discovery_doc, developerKey=api_key
)
def create_file(
self,
path: str | pathlib.Path | os.PathLike,
*,
mime_type: str | None = None,
name: str | None = None,
display_name: str | None = None,
resumable: bool = True,
metadata: Sequence[tuple[str, str]] = (),
) -> protos.File:
self._setup_discovery_api(metadata)
file = {}
if name is not None:
file["name"] = name
if display_name is not None:
file["displayName"] = display_name
media = googleapiclient.http.MediaFileUpload(
filename=path, mimetype=mime_type, resumable=resumable
)
request = self._local.discovery_api.media().upload(body={"file": file}, media_body=media)
for key, value in metadata:
request.headers[key] = value
result = request.execute()
return self.get_file({"name": result["file"]["name"]})
@MarkDaoust this is working for me with your suggestion for anyones reference
@MarkDaoust I've raised it here https://github.com/google-gemini/generative-ai-python/pull/583
