genkit icon indicating copy to clipboard operation
genkit copied to clipboard

[JS] [dotprompt] Must supply a `contentType` when sending File URIs to Gemini.

Open odbol opened this issue 1 year ago • 3 comments

Describe the bug

Using the example in the docs for multimodal, I get this error:

Must supply a `contentType` when sending File URIs to Gemini.
Error: Must supply a `contentType` when sending File URIs to Gemini.
    at toFileData (/home/user/gemineye-server/node_modules/@genkit-ai/googleai/lib/gemini.js:255:11)
    at toGeminiPart (/home/user/gemineye-server/node_modules/@genkit-ai/googleai/lib/gemini.js:369:12)
    at Array.map (<anonymous>)
    at toGeminiMessage (/home/user/gemineye-server/node_modules/@genkit-ai/googleai/lib/gemini.js:400:28)
    at /home/user/gemineye-server/node_modules/@genkit-ai/googleai/lib/gemini.js:532:19
    at Generator.next (<anonymous>)
    at /home/user/gemineye-server/node_modules/@genkit-ai/googleai/lib/gemini.js:66:61
    at new Promise (<anonymous>)
    at __async (/home/user/gemineye-server/node_modules/@genkit-ai/googleai/lib/gemini.js:50:10)
    at /home/user/gemineye-server/node_modules/@genkit-ai/googleai/lib/gemini.js:479:37

To Reproduce

---
model: googleai/gemini-1.5-pro-latest
config:
  temperature: 0.85
input:
  schema:
    imageUrl: string
---
{{role "system"}}
You are a helpful screen reader assistant for blind people.
{{role "user"}}
Describe the image

{{media url=photoUrl}}

Expected behavior Should send the image. Or docs should mention that you need to specify the contentType... somehow (although isn't that supposed to be in the data uri?)

Screenshots If applicable, add screenshots to help explain your problem.

Runtime (please complete the following information):

  • OS: Linux
  • Version [e.g. 22]

** Node version

  • run node --version at paste here 20

Additional context Add any other context about the problem here.

odbol avatar Aug 31 '24 00:08 odbol

+1 on this, same error. The example in the docs does not work.

danwritecode avatar Nov 16 '24 16:11 danwritecode

I used Firebase Studio to create an app with a prompt and even that application gets this error thrown.

moekhalil avatar Apr 13 '25 12:04 moekhalil

You need to specify the contentType (mime type) as indicated in the error message. (e.g image/png)

---
model: googleai/gemini-2.0-flash
config:
  temperature: 0.85
input:
  schema:
    imageUrl: string
    mimeType: string
---
{{role "system"}}
You are a helpful screen reader assistant for blind people.
{{role "user"}}
Describe the image

{{media url=imageUrl contentType=mimeType}}

kenryu42 avatar May 11 '25 15:05 kenryu42