inference llama-7b error

[Chevalier1024]

response is none and the generated_token_ids are all 7

[Ericmututu]

same error!!! Have you solved it yet?

[hychaochao]

I encountered the same error when using alpaca-lora for inference. In the end, it was because using bitsandbytes to quantify the model failed. I wonder if this is the problem for you.