Enabling JSON Format Responses for Image Inputs in Gemini 1.5

[kihapper]

trafficstars

The new Gemini 1.5 model has the capability to enforce JSON responses, but there is limited documentation available on how to implement this, particularly for obtaining JSON responses from image inputs.

Could you provide an example of how to achieve this? The REST example here and the official doc provided did not give me clarity on the process.

[kihapper]

I am calling it like below in python but is getting the error message ValueError: Protocol message GenerationConfig has no "response_mime_type" field.

# Configure JSON response and safety settings
generation_config = {
    'response_mime_type': 'application/json'  # Add this line for JSON response
}

# Call generate_content with the updated configuration
response = model.generate_content(
    [prompt_variable, image_data],
    generation_config=generation_config,
    stream=False
)

 response.resolve()

[TYMichaelChen]

Hi @kihapper thanks for reporting the issue. We are working on this - This should be fixed in the upcoming SDK release. Will update this once the SDK is published.

[kihapper]

@TYMichaelChen Thanks for this! Do you know roughly when these SDK will be rolled out for python and node.js?

[markmcd]

Like Michael says - we're still working on the feature, though some support in the Python SDK is available at HEAD. You can pip install git+https://github.com/google/generative-ai-python to get the latest version and try it out, but caveat emptor.

For the specific proto field you're trying to use, as long as you have google-ai-generativelanguage>=0.6.2, you should be good to go.

[TYMichaelChen]

The newest SDK on PyPI should now have the changes: https://pypi.org/project/google-generativeai/0.5.1/. Let us know if you are still seeing this issue