OpenBMB
[BUG] web_demo_2.6.py RuntimeError: probability tensor contains either `inf`, `nan` or element < 0
是否已有关于该错误的issue或讨论？ | Is there an existing issue / discussion for this?
- [X] 我已经搜索过已有的issues和讨论 | I have searched the existing issues / discussions
该问题是否在FAQ中有解答？ | Is there an existing answer for this in FAQ?
- [X] 我已经搜索过FAQ | I have searched FAQ
当前行为 | Current Behavior
python web_demo_2.6.py --device cuda
msgs: [{'role': 'user', 'content': [<PIL.Image.Image image mode=RGB size=1000x749 at 0x7FAB0BDEA650>, '图片中是不是存在暴力行为']}]
/data/miniconda3/envs/MiniCPM-V/lib/python3.10/site-packages/huggingface_hub/file_download.py:1150: FutureWarning: resume_download is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use force_download=True.
warnings.warn(
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
probability tensor contains either inf, nan or element < 0
Traceback (most recent call last):
File "/data/MiniCPM-V/web_demo_2.6.py", line 134, in chat
answer = model.chat(
File "/data/.cache/huggingface/modules/transformers_modules/openbmb/MiniCPM-V-2_6/a38652350cf48f3cedc8e8b2ff73801fe2dcc004/modeling_minicpmv.py", line 380, in chat
res = self.generate(
File "/data/.cache/huggingface/modules/transformers_modules/openbmb/MiniCPM-V-2_6/a38652350cf48f3cedc8e8b2ff73801fe2dcc004/modeling_minicpmv.py", line 261, in generate
result = self._decode(model_inputs["inputs_embeds"], tokenizer, attention_mask, decode_text=decode_text, **kwargs)
File "/data/.cache/huggingface/modules/transformers_modules/openbmb/MiniCPM-V-2_6/a38652350cf48f3cedc8e8b2ff73801fe2dcc004/modeling_minicpmv.py", line 185, in _decode
output = self.llm.generate(
File "/data/miniconda3/envs/MiniCPM-V/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/data/miniconda3/envs/MiniCPM-V/lib/python3.10/site-packages/transformers/generation/utils.py", line 1622, in generate
result = self._sample(
File "/data/miniconda3/envs/MiniCPM-V/lib/python3.10/site-packages/transformers/generation/utils.py", line 2829, in _sample
next_tokens = torch.multinomial(probs, num_samples=1).squeeze(1)
RuntimeError: probability tensor contains either inf, nan or element < 0
期望行为 | Expected Behavior
No response
复现方法 | Steps To Reproduce
No response
运行环境 | Environment
- OS: Ubuntu 22.04
- Python: 3.10.14
- Transformers: 4.40.0
- PyTorch: 2.1.2
- CUDA (`python -c 'import torch; print(torch.version.cuda)'`): 12.1
备注 | Anything else?
No response
请确认web_demo_2.6.py代码没有做任何改动，模型权重是否下载完整(可以检查sha256值跟hf上是否一致)； 如果可以麻烦提供可复现的测试原图和prompt
模型正常，使用原始模型可以正常推理，微调过后出现 RuntimeError: probability tensor contains either inf, nan or element < 0
