Binyuan Hui
Binyuan Hui
I tried to align vllm with the inference of huggingface and found some diffs, the debugging code is as follows. test file: ``` from vllm import LLM llm = LLM(model="OLMoE/OLMoE-1B-7B-0824",...
@janimo 🤔 Could you have some debugging info to verify this difference? Contributions are very welcome!
@mgoin Thanks for taking the time to help us debug! But unfortunately at the moment the simplest prompt still don't respond correctly in vllm, so reproducing scores from benchmarks is...
https://github.com/QwenLM/Qwen2.5-Coder/tree/main/qwencoder-eval/instruct/bird-spider
 你需要拖动具体日期，与我们表格中的日期对应。
For most evaluations, we adopt a greedy decoding. You can find all the evaluation details in our evaluation scripts. https://github.com/QwenLM/Qwen2.5-Coder/tree/main/qwencoder-eval/instruct