lmdeploy
lmdeploy copied to clipboard
InternLM
lmdeploy搭建的服务，是否支持通过传输stop_words的方式来控制模型输出
Checklist
- [ ] 1. I have searched related issues but cannot get the expected help.
- [ ] 2. The bug has not been fixed in the latest version.
Describe the bug
搭建服务：lmdeploy serve api_server 请求：基于openai的协议进行请求 client.chat.completions.create(model=model,messages=messages,functions=functions,stop=STOP_WORDS)
传入的STOP_WORDS没有生效
Reproduction
无
Environment
lmdeploy：0.4.0
Error traceback
No response
每个stop_word经过decode后，有几个token id呢？ 目前 lmdeploy 只能支持单 token 的 stop_word
加入的strop_word 经过decode后对应两个token id。
那如果我直接修改对模板中的stop_word能够生效吗？比如将下面的对话那边中的stop_words对应的值改为我们这边指定的stop word
{ "model_name": "your awesome chat template name", "system": "<|im_start|>system\n", "meta_instruction": "You are a robot developed by LMDeploy.", "eosys": "<|im_end|>\n", "user": "<|im_start|>user\n", "eoh": "<|im_end|>\n", "assistant": "<|im_start|>assistant\n", "eoa": "<|im_end|>", "separator": "\n", "capability": "chat", "stop_words": ["<|im_end|>"] }