probability tensor contains either `inf`, `nan` or element < 0

[Minami-su]

CUDA_VISIBLE_DEVICES=0 python llama_inference.py decapoda-research/llama-7b-hf --wbits 4 --load llama7b-4bit.pt --text "this is llama" Loading model ... Done. Traceback (most recent call last): File "llama_inference.py", line 115, in generated_ids = model.generate( File "/root/miniconda3/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context return func(*args, **kwargs) File "/root/miniconda3/lib/python3.8/site-packages/transformers-4.27.0.dev0-py3.8.egg/transformers/generation/utils.py", line 1452, in generate return self.sample( File "/root/miniconda3/lib/python3.8/site-packages/transformers-4.27.0.dev0-py3.8.egg/transformers/generation/utils.py", line 2504, in sample next_tokens = torch.multinomial(probs, num_samples=1).squeeze(1) RuntimeError: probability tensor contains either inf, nan or element < 0