LLaVA ERROR when running the CLI.

Discussion

Good afternoon everybody!

I'm Paulo Roberto, born in Rio de Janeiro - Brazil

I really need help.

https://github.com/haotian-liu/LLaVA?tab=readme-ov-file#Demo

python -m llava.serve.cli
--model-path liuhaotian/llava-v1.5-7b
--image-file "https://llava-vl.github.io/static/images/view.jpg"
--load-4bit

My environment is WSL2 (Windows 11)

The mistake

(llava) prsantos@LAPTOP-940KNJ7R:~/Desenvolvimentos/ProjetosPython/ProjetosITTech/LLaVA$ python -m llava.serve.cli
--model-path liuhaotian/llava-v1.5-7b
--image-file "https://llava-vl.github.io/static/images/view.jpg"
--load-4bit Traceback (most recent call last): File "", line 198, in _run_module_as_main File "", line 88, in _run_code File "/home/prsantos/Desenvolvimentos/ProjetosPython/ProjetosITTech/LLaVA/llava/serve/cli.py", line 128, in main(args) File "/home/prsantos/Desenvolvimentos/ProjetosPython/ProjetosITTech/LLaVA/llava/serve/cli.py", line 32, in main tokenizer, model, image_processor, context_len = load_pretrained_model(args.model_path, args.model_base, model_name, args.load_8bit, args.load_4bit, device=args.device) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^ File "/home/prsantos/Desenvolvimentos/ProjetosPython/ProjetosITTech/LLaVA/llava/model/builder.py", line 116, in load_pretrained_model tokenizer = AutoTokenizer.from_pretrained(model_path, use_fast=False) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^ File "/home/prsantos/miniconda3/envs/llava/lib/python3.11/site-packages/transformers/models/auto/tokenization_auto.py", line 814, in from_pretrained return tokenizer_class.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/prsantos/miniconda3/envs/llava/lib/python3.11/site-packages/transformers/tokenization_utils_base.py", line 2029, in from_pretrained return cls._from_pretrained( ^^^^^^^^^^^^^^^^^^^^^ File "/home/prsantos/miniconda3/envs/llava/lib/python3.11/site-packages/transformers/tokenization_utils_base.py", line 2261, in _from_pretrained tokenizer = cls(*init_inputs, **init_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/prsantos/miniconda3/envs/llava/lib/python3.11/site-packages/transformers/models/llama/tokenization_llama.py", line 178, in init self.sp_model = self.get_spm_processor(kwargs.pop("from_slow", False)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^ File "/home/prsantos/miniconda3/envs/llava/lib/python3.11/site-packages/transformers/models/llama/tokenization_llama.py", line 208, in get_spm_processor model_pb2 = import_protobuf(f"The new behavior of {self.class.name} (with self.legacy = False)") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/prsantos/miniconda3/envs/llava/lib/python3.11/site-packages/transformers/convert_slow_tokenizer.py", line 43, in import_protobuf raise ImportError(PROTOBUF_IMPORT_ERROR.format(error_message)) ImportError: The new behavior of LlamaTokenizer (with self.legacy = False) requires the protobuf library but it was not found in your environment. Checkout the instructions on the installation page of its repo: https://github.com/protocolbuffers/protobuf/tree/master/python#installation and follow the ones that matches your environment. Please note that you may need to restart your runtime after installation.

Can you help me please?

I need to use LLaVA to analyze several images using a Python script.

I'm having a lot of trouble getting this result.

Thank you in advance for the feedback.