When using Python 3.11.4 to invoke the Gemini Pro Vision model and upload a plt image, a type error is reported.

[onethefly]

Description of the bug:

! curl -o image.jpg https://t0.gstatic.com/licensed-image?q=tbn:ANd9GcQ_Kevbk21QBRy-PgB4kQpS79brbmmEG7m3VOTShAn4PecDU5H5UxrJxE3Dw1JiaG17V88QIol19-3TM2wCHw

img = PIL.Image.open('image.jpg') model = genai.GenerativeModel('gemini-pro-vision') response = model.generate_content(img)

---

TypeError Traceback (most recent call last) Cell In[56], line 1 ----> 1 response = model.generate_content(img)

File ~/.local/lib/python3.11/site-packages/google/generativeai/generative_models.py:229, in GenerativeModel.generate_content(self, contents, generation_config, safety_settings, stream, **kwargs) 219 @string_utils.set_doc(_GENERATE_CONTENT_DOC) 220 def generate_content( 221 self, (...) 227 **kwargs, 228 ) -> generation_types.GenerateContentResponse: --> 229 request = self._prepare_request( 230 contents=contents, 231 generation_config=generation_config, 232 safety_settings=safety_settings, 233 **kwargs, 234 ) 235 if self._client is None: 236 self._client = client.get_default_generative_client()

File ~/.local/lib/python3.11/site-packages/google/generativeai/generative_models.py:200, in GenerativeModel._prepare_request(self, contents, generation_config, safety_settings, **kwargs) 197 if not contents: 198 raise TypeError("contents must not be empty") --> 200 contents = content_types.to_contents(contents) 202 generation_config = generation_types.to_generation_config_dict(generation_config) 203 merged_gc = self._generation_config.copy()

File ~/.local/lib/python3.11/site-packages/google/generativeai/types/content_types.py:235, in to_contents(contents) 230 except TypeError: 231 # If you get a TypeError here it's probably because that was a list 232 # of parts, not a list of contents, so fall back to to_content. 233 pass --> 235 contents = [to_content(contents)] 236 return contents

File ~/.local/lib/python3.11/site-packages/google/generativeai/types/content_types.py:201, in to_content(content) 198 return glm.Content(parts=[to_part(part) for part in content]) 199 else: 200 # Maybe this is a Part? --> 201 return glm.Content(parts=[to_part(content)])

File ~/.local/lib/python3.11/site-packages/google/generativeai/types/content_types.py:171, in to_part(part) 168 return glm.Part(text=part) 169 else: 170 # Maybe it can be turned into a blob? --> 171 return glm.Part(inline_data=to_blob(part))

File ~/.local/lib/python3.11/site-packages/google/generativeai/types/content_types.py:140, in to_blob(blob) 136 if isinstance(blob, Mapping): 137 raise KeyError( 138 "Could not recognize the intended type of the dict\n" "A content should have " 139 ) --> 140 raise TypeError( 141 "Could not create Blob, expected Blob, dict or an Image type" 142 "(PIL.Image.Image or IPython.display.Image).\n" 143 f"Got a: {type(blob)}\n" 144 f"Value: {blob}" 145 )

TypeError: Could not create Blob, expected Blob, dict or an Image type(PIL.Image.Image or IPython.display.Image). Got a: <class 'PIL.JpegImagePlugin.JpegImageFile'> Value: <PIL.JpegImagePlugin.JpegImageFile image mode=RGB size=2048x1365 at 0x7F89DDFC8B10>

Actual vs expected behavior:

No response

Any other information you'd like to share?

python 3.11.4 centos 7.9 google-ai-generativelanguage 0.4.0 google-api-core 2.15.0 google-auth 2.25.2 google-generativeai 0.3.1 googleapis-common-protos 1.62.0 langchain-google-genai 0.0.4

[jhowardsemail]

Try using IPython.display instead of PIL for Image. Examples:

from IPython.display import Image from IPython.core.display import HTML Image(url= "http://my_site.com/my_picture.jpg")

or from IPython.display import Image from IPython.core.display import HTML img = Image('image_test.jpg')

[crozzdev]

Thanks @jhowardsemail It worked for me!

[Neon2k2]

Thank you @jhowardsemail worked for me as well!!

[MarkDaoust]

I'm unable to reproduce this error now. I think it's fixed. If not please reopen and send code to reproduce the problem.

Thanks.

[MadCoderme]

@MarkDaoust I am getting similar error but for counting tokens:

result = model.generate_content(prompt)
token_count = model.count_tokens(code).total_tokens

This is the error message I get

Could not create `Blob`, expected `Blob`, `dict` or an `Image` type(`PIL.Image.Image` or `IPython.display.Image`).
Got a: <class 'int'>
Value: 35

(I'm using tuned model if that matters)

[Siddharth-Xenon]

@MadCoderme Did you find the solution?

[MarkDaoust]

We did have an issue where a lot of these were popping up in conda. Are either of you using conda?

@MadCoderme "Got a: <class 'int'>" It's trying to tell you that it doesn't know how to turn an int into a prompt. Why are you passing it an int?

[ashu5711]

Through Ipython it helps alternatively you can upload_file to Geminipro also.