miaoyuxun

如何使vllm下测评结果更接近hf？

2

vllm中的测评结果在各个模型上都比hf差一或两个点 评测代码逻辑大致如下： hf ``` import json with open('diff_inputs.json','r',encoding='utf-8') as file: inputs = json.load(file) from transformers import AutoModelForCausalLM,AutoTokenizer import torch model = AutoModelForCausalLM.from_pretrained( ".../llama-2-7B" ).bfloat16() tokenizer = AutoTokenizer.from_pretrained(".../llama-2-7B") if tokenizer.pad_token is...