tianshiyisi

function call 如何返回usage相关的信息

2

有什么办法能统计function call调用过程中输入和输出的token数吗，试了一下修改oai.py代码，打印出的chunk里也没有相关信息 root@vllm:~/benchmark# python3 fcall.py 请输入查询内容（输入 'quit' 退出）：你好 ChatCompletionChunk(id='chat-a55bee23999c4e37ae2bc50b1bf4b491', choices=[Choice(delta=ChoiceDelta(content='', function_call=None, refusal=None, role='assistant', tool_calls=None), finish_reason=None, index=0, logprobs=None)], created=1726676479, model='vllm-qwen2-plus', object='chat.completion.chunk', service_tier=None, system_fingerprint=None, usage=None) ChatCompletionChunk(id='chat-a55bee23999c4e37ae2bc50b1bf4b491', choices=[Choice(delta=ChoiceDelta(content='', function_call=None, refusal=None, role=None, tool_calls=None), finish_reason=None,...

[Badcase]: qwen2.5-72b 在昇腾910推理结果不符合预期

8

### Model Series Qwen2.5 ### What are the models used? qwen2.5-72b-instruct ### What is the scenario where the problem happened? qwen2.5-72b-instruct 在昇腾910b上推理异常 ### Is this badcase known and can it...