module 'mediapipe.python._framework_bindings.model_ckpt_util' has no attribute 'ConvertHfTokenizer'

[mohamedlotfy50]

I am to convert a Gemma model to TFLite for CPU inference following the genai documentation. However, I'm encountering an error:

module 'mediapipe.python._framework_bindings.model_ckpt_util' has no attribute 'ConvertHfTokenizer'

Here is my code:

import mediapipe as mp
from mediapipe.tasks.python.genai import converter

def gemma_convert_config(backend):
  """Configures conversion for Gemma model to TFLite."""
  input_ckpt = 'gemma_2b_it/'
  vocab_model_file = 'gemma_2b_it/'
  output_dir = 'gemma_2b/'
  output_tflite_file = f'gemma_2b/gemma_{backend}.bin'
  return converter.ConversionConfig(
      input_ckpt=input_ckpt,
      ckpt_format='safetensors',
      model_type='GEMMA_2B',
      backend=backend,
      output_dir=output_dir,
      combine_file_only=True,
      vocab_model_file=vocab_model_file,
      output_tflite_file=output_tflite_file)

config = gemma_convert_config('cpu')

converter.convert_checkpoint(config)

MediaPipe Version: 0.10.14

[hheydary]

Greetings,

I am not able to reproduce this issue. Please consider using the example colab provided here for converting models and learning more about the required arguments to the converter.

[shlok191]

Hello, I am seeing the same error. I am running the colab notebook on my local conda environment with python=3.10. EDIT: I was able to get the software working on a Linux machine. Hopefully this helps for the devs!

[kuaashish]

Hi @mohamedlotfy50,

Could you please specify the environment in which you are encountering this error? For example, the operating system, Python version, whether you're using Colab, or if you are using a conda environment. This information will help us reproduce and understand the issue better.

Thank you!!

[mohamedlotfy50]

Hello @kuaashish

These are the specifications that I am currently running:

• Operating system: Windows 11
• Python: 3.11.4
• environment: conda

Thank you

[kuaashish]

Hi @mohamedlotfy50,

Thank you for the update. It appears to be a bug in the Windows environment. We are sharing this issue with the team and hope to get a fix soon.

[google-ml-butler[bot]]

Are you satisfied with the resolution of your issue? Yes No

[schmidt-sebastian]

Sorry - we do not support Windows for our GenAI tasks at this moment as we do not have the GPU support on Windows at the moment. We hope that we will have a better answer soon.

[google-ml-butler[bot]]

Are you satisfied with the resolution of your issue? Yes No

[itzjac]

Excuse me, can't find any specifics on what OS is suppose to be used to execute the convertion? All the threads mention support on Windows. Should this be run exclusively in a linux enviroment?