Baichuan2
Baichuan2 copied to clipboard
device_map为"auto"和"cuda:0"时,model.generate_config加载结果不一致
auto: { "bos_token_id": 1, "pad_token_id": 0, "eos_token_id": 2, }
cuda:0: { "assistant_token_id": 196, "bos_token_id": 1, "do_sample": true, "eos_token_id": 2, "max_new_tokens": 2048, "pad_token_id": 0, "repetition_penalty": 1.05, "temperature": 0.1, "top_k": 3, "top_p": 0.15, "transformers_version": "4.31.0", "user_token_id": 195 }
Thank you for your feedback. We will follow up on this issue.