LLaVA icon indicating copy to clipboard operation
LLaVA copied to clipboard

[Usage] The lora models could not implement at the latest version of LLaVA

Open y-vectorfield opened this issue 2 years ago • 0 comments

When I implemented the lora models(llava-v1.5-13b-lora, llava-v1.5-7b-lora), the following error was occurred.

/usr/local/lib/python3.10/dist-packages/llava/model/builder.py:48: UserWarning: There is `lora` in model name but no `model_base` is provided. If you are loading a LoRA model, please provide the `model_base` argument. Detailed instruction: https://github.com/haotian-liu/LLaVA#launch-a-model-worker-lora-weights-unmerged.
  warnings.warn('There is `lora` in model name but no `model_base` is provided. If you are loading a LoRA model, please provide the `model_base` argument. Detailed instruction: https://github.com/haotian-liu/LLaVA#launch-a-model-worker-lora-weights-unmerged.')
config.json: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1.02k/1.02k [00:00<00:00, 4.63MB/s]
Traceback (most recent call last):
  File "/lustre/groups/share/yvectorfield/ai-models/llava/common/eval_llava.py", line 106, in <module>
    main(args)
  File "/lustre/groups/share/yvectorfield/ai-models/llava/common/eval_llava.py", line 87, in main
    eval_model(model_args)
  File "/usr/local/lib/python3.10/dist-packages/llava/eval/run_llava.py", line 56, in eval_model
    tokenizer, model, image_processor, context_len = load_pretrained_model(
  File "/usr/local/lib/python3.10/dist-packages/llava/model/builder.py", line 105, in load_pretrained_model
    tokenizer = AutoTokenizer.from_pretrained(model_path, use_fast=False)
  File "/usr/local/lib/python3.10/dist-packages/transformers/models/auto/tokenization_auto.py", line 718, in from_pretrained
    tokenizer_class_py, tokenizer_class_fast = TOKENIZER_MAPPING[type(config)]
  File "/usr/local/lib/python3.10/dist-packages/transformers/models/auto/auto_factory.py", line 663, in __getitem__
    model_type = self._reverse_config_mapping[key.__name__]
KeyError: 'LlavaConfig'

y-vectorfield avatar Jan 30 '24 06:01 y-vectorfield