gemma icon indicating copy to clipboard operation
gemma copied to clipboard

NotImplementedError: Multi-turn with images on the second turn is not supported yet

Open radna0 opened this issue 6 months ago • 1 comments

# Common imports
import os
import jax.numpy as jnp
import tensorflow_datasets as tfds

# Gemma imports
from gemma import gm


os.environ["XLA_PYTHON_CLIENT_MEM_FRACTION"] = "1.00"

ds = tfds.data_source("oxford_flowers102", split="train")
image1 = ds[0]["image"]
image2 = ds[1]["image"]

model = gm.nn.Gemma3_4B()

params = gm.ckpts.load_params(gm.ckpts.CheckpointPath.GEMMA3_4B_IT)

sampler = gm.text.ChatSampler(
    model=model,
    params=params,
    multi_turn=True,
)


out = sampler.chat(
    "What can you say about this image: <start_of_image>",
    images=image1,
)

out1 = sampler.chat(
    "What about this other image?: <start_of_image>",
    images=image2,
)

radna0 avatar Jun 25 '25 23:06 radna0

Hey, I recently updated the sampler to fix this use-case so it should work now. However I did not released it yet on PyPI.

You could try to pip install git+https://github.com/google-deepmind/gemma to get the latest code.

pip install git+https://github.com/google/etils
pip install git+https://github.com/google-research/kauldron
pip install git+https://github.com/google-deepmind/gemma

This should install the last version of Gemma as well as it's associated deps

Conchylicultor avatar Jul 02 '25 12:07 Conchylicultor