InternVL ERROR when eval internvl_chat

Hi! Thanks for the great work. I am working on the multi-model Vision-QA tasks using internvl_chat_llava model. When evaluate the internvl_chat_llava model, an error occurs:

Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/workspace/code/InternVL/internvl_chat_llava/llava/eval/model_vqa_loader.py", line 215, in <module>
    eval_model(args)
  File "/workspace/code/InternVL/internvl_chat_llava/llava/eval/model_vqa_loader.py", line 173, in eval_model
    output_ids = model.generate(
  File "/usr/local/lib/python3.8/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/transformers/generation/utils.py", line 1530, in generate
    self._validate_model_kwargs(model_kwargs.copy())
  File "/usr/local/lib/python3.8/dist-packages/transformers/generation/utils.py", line 1344, in _validate_model_kwargs
    raise ValueError(
ValueError: The following `model_kwargs` are not used by the model: ['images'] (note: typos in the generate arguments will also show up in this list)

It seems like in

        with torch.inference_mode():
            output_ids = model.generate(
                input_ids,
                images=image_tensor.to(dtype=torch.float16, device='cuda', non_blocking=True),
                do_sample=True if args.temperature > 0 else False,
                temperature=args.temperature,
                top_p=args.top_p,
                num_beams=args.num_beams,
                max_new_tokens=128,
                use_cache=True)

the images tensors should not be passed to the model.generate function. But I don't know why. May I have some help? Looking forward to your reply! @czczup @shepnerd

Apr 16 '24 07:04 dszpr

My model_path is the OUTPUT_DIR of the fituned internvit6b_224to336_vicuna7b model. Does the error have something to do with the fituned model? It really confused me. Everything went right when finetuning.

Apr 16 '24 08:04 dszpr

Has this problem been solved? If not, may I ask what version of transformers you are using.

Apr 27 '24 13:04 czczup

The problem has been solved after updating transformers. Thanks.

May 23 '24 08:05 dszpr

ERROR when eval internvl_chat_llava model