V100推理internVL-1.5-Int8问题

[rTrQqgH74lc2PT5k]

Describe the bug What the bug is, and how to reproduce, better with screenshots(描述bug以及复现过程，最好有截图)

1. 使用最新代码后，使用int8模型后会报错 RuntimeError: probability tensor contains either inf, nan or element < 0
2. 如果不使用命令行，使用推理脚本 则推理出来后response是乱码

Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
FlashAttention is not installed.
Unused kwargs: ['quant_method']. These kwargs are not used in <class 'transformers.utils.quantization_config.BitsAndBytesConfig'>.
Loading checkpoint shards: 100%|██████████████████████| 6/6 [00:58<00:00,  9.77s/it]
[INFO:swift] model.max_model_len: None
[INFO:swift] Global seed set to 42
query: How far is it from each city?
response:  <unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk>
query: Which city is the farthest?
response: The data</s>
history: [['How far is it from each city?', ' <unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk>], ['Which city is the farthest?', 'The data</s>']]

Your hardware and system info

Pytorch 2.2, CUDA 11.8