RuntimeError: probability tensor contains either `inf`, `nan` or element < 0

[le-wei]

Describe the bug When deploying MiniGPT-v2, I encountered the following error while running the example:

Could not create share link. Please check your internet connection or our status page: https://status.gradio.app. Exception in thread Thread-8: Traceback (most recent call last): File "/home/ubuntu/anaconda3/envs/miniGpt/lib/python3.9/threading.py", line 980, in _bootstrap_inner self.run() File "/home/ubuntu/anaconda3/envs/miniGpt/lib/python3.9/threading.py", line 917, in run self._target(*self._args, **self._kwargs) File "/home/ubuntu/objectData/MiniGPT-4/minigpt4/conversation/conversation.py", line 207, in model_generate output = self.model.llama_model.generate(*args, **kwargs) File "/home/ubuntu/anaconda3/envs/miniGpt/lib/python3.9/site-packages/peft/peft_model.py", line 580, in generate return self.base_model.generate(**kwargs) File "/home/ubuntu/anaconda3/envs/miniGpt/lib/python3.9/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(*args, **kwargs) File "/home/ubuntu/anaconda3/envs/miniGpt/lib/python3.9/site-packages/transformers/generation/utils.py", line 1572, in generate return self.sample( File "/home/ubuntu/anaconda3/envs/miniGpt/lib/python3.9/site-packages/transformers/generation/utils.py", line 2655, in sample next_tokens = torch.multinomial(probs, num_samples=1).squeeze(1) RuntimeError: probability tensor contains either inf, nan or element < 0

Screenshots

Desktop (please complete the following information):

• OS: [e.g.Ubuntu 20.4]
• Browser [e.g. chrome]
• Version [e.g. 22]

Additional context

Please help me, thank you.

[junchen14]

this has never happens to us before when the batch size=1. Does this happen frequently from your side?

[le-wei]

I have been encountering this issue constantly and have not yet succeeded in deducing the cause. I am not sure if there is a problem with my configuration, or from what aspect I should troubleshoot. Thank you very much for your reply.

[ChaosPengs]

did you solve it yet?