[Bad Case]: 多模态MiniCPM-V 2.0 transformers 推理报错

[wangyao123456a]

Description / 描述

import torch from PIL import Image from transformers import AutoModel, AutoTokenizer

model = AutoModel.from_pretrained('./model/OpenBMB/MiniCPM-V-2', trust_remote_code=True)

model = model.to(device='cuda')

tokenizer = AutoTokenizer.from_pretrained('./model/OpenBMB/MiniCPM-V-2', trust_remote_code=True) model.eval()

image = Image.open('./img/tmp.jpg').convert('RGB') question = 'What is in the image?' msgs = [{'role': 'user', 'content': question}]

res, context, _ = model.chat( image=image, msgs=msgs, context=None, tokenizer=tokenizer, sampling=True, temperature=0.7 ) print(res) 推理报错： File "/mnt/data1/wy/MiniCPM-V/predict_v2.py", line 22, in res, context, _ = model.chat( File "/root/.cache/huggingface/modules/transformers_modules/MiniCPM-V-2/modeling_minicpmv.py", line 324, in chat content = final_placeholder + "\n" + content TypeError: can only concatenate str (not "list") to str (bert) [\e]0;\u@\h: \w\a]\u@\h:\w$ python predict_v2.py Loading checkpoint shards: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 15.07it/s] The seen_tokens attribute is deprecated and will be removed in v4.41. Use the cache_position model input instead. Traceback (most recent call last): File "/mnt/data1/wy/MiniCPM-V/predict_v2.py", line 22, in res, context, _ = model.chat( File "/root/.cache/huggingface/modules/transformers_modules/MiniCPM-V-2/modeling_minicpmv.py", line 358, in chat res, vision_hidden_states = self.generate( File "/root/.cache/huggingface/modules/transformers_modules/MiniCPM-V-2/modeling_minicpmv.py", line 288, in generate result = self._decode(model_inputs["inputs_embeds"], tokenizer, **kwargs) File "/root/.cache/huggingface/modules/transformers_modules/MiniCPM-V-2/modeling_minicpmv.py", line 192, in _decode output = self.llm.generate( File "/usr/local/anaconda3/envs/bert/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context return func(*args, **kwargs) File "/usr/local/anaconda3/envs/bert/lib/python3.10/site-packages/transformers/generation/utils.py", line 1989, in generate result = self._sample( File "/usr/local/anaconda3/envs/bert/lib/python3.10/site-packages/transformers/generation/utils.py", line 2932, in _sample outputs = self(**model_inputs, return_dict=True) File "/usr/local/anaconda3/envs/bert/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl return self._call_impl(*args, **kwargs) File "/usr/local/anaconda3/envs/bert/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl return forward_call(*args, **kwargs) File "/root/.cache/huggingface/modules/transformers_modules/MiniCPM-V-2/modeling_minicpm.py", line 1387, in forward outputs = self.model( File "/usr/local/anaconda3/envs/bert/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl return self._call_impl(*args, **kwargs) File "/usr/local/anaconda3/envs/bert/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl return forward_call(*args, **kwargs) File "/root/.cache/huggingface/modules/transformers_modules/MiniCPM-V-2/modeling_minicpm.py", line 1262, in forward layer_outputs = decoder_layer( File "/usr/local/anaconda3/envs/bert/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl return self._call_impl(*args, **kwargs) File "/usr/local/anaconda3/envs/bert/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl return forward_call(*args, **kwargs) File "/root/.cache/huggingface/modules/transformers_modules/MiniCPM-V-2/modeling_minicpm.py", line 973, in forward hidden_states, self_attn_weights, present_key_value = self.self_attn( File "/usr/local/anaconda3/envs/bert/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl return self._call_impl(*args, **kwargs) File "/usr/local/anaconda3/envs/bert/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl return forward_call(*args, **kwargs) File "/root/.cache/huggingface/modules/transformers_modules/MiniCPM-V-2/modeling_minicpm.py", line 869, in forward query_states, key_states = apply_rotary_pos_emb( File "/root/.cache/huggingface/modules/transformers_modules/MiniCPM-V-2/modeling_minicpm.py", line 284, in apply_rotary_pos_emb cos = cos[position_ids].unsqueeze(unsqueeze_dim) # [bs, 1, seq_len, dim] IndexError: index is out of bounds for dimension with size 0

Case Explaination / 案例解释

服务器环境是4090

[c122-ode]

MiniCPM-V i found output = self.llm.generate( inputs_embeds=inputs_embeds, pad_token_id=0, eos_token_id=tokenizer.eos_token_id, **kwargs ) "inputs_embeds " not None But when met "forward": MiniCPMForCausalLM :forward() inputs_embeds is None.

what happens in the process of "self.llm.generate"

[PangziZhang523]

请问有解决嘛，我也遇到这个问题

[vutrongnghiahaui]

I also got the same error with ubuntu server 24.04. but with ubuntu server 22.04 there is no problem. is it possible that the error is due to the operating system.

[PangziZhang523]

我把transformer的版本换到transformers==4.40.0，解决了

[vutrongnghiahaui]

我把transformer的版本换到transformers==4.40.0，解决了

what ubuntu do you use? As I tested, the latest version of transformers still runs normally on ubuntu 22.04. I don't want to downgrade the version of transformers.