Simon Mo

I think the test failure in the entrypoints test might be related, and there's merge conflict. 🙏

Getting ``` Traceback (most recent call last): File "/opt/conda/lib/python3.10/runpy.py", line 196, in _run_module_as_main return _run_code(code, main_globals, None, File "/opt/conda/lib/python3.10/runpy.py", line 86, in _run_code exec(code, run_globals) File "/home/xmo/vllm/vllm/entrypoints/openai/api_server.py", line 157, in...

I'm running into the following issues: * Completion not working * Chat template is missing in tokenizer config, the default one will just keep the generation going forever without EOS....

The tokenizer_config.json should also include the following so it doesn't need to be specified by the client each time. Please let me know once the hf or modelscope version is...

Looks good, please fix lint by running ./format.sh

https://github.com/vllm-project/vllm/actions/runs/8720404543/job/23921845033?pr=4078#step:5:1 ``` Run yapf --diff --recursive . --- ./vllm/model_executor/models/zhinao.py (original) +++ ./vllm/model_executor/models/zhinao.py (reformatted) @@ -327,7 +327,9 @@ super().__init__() self.config = config self.linear_method = linear_method - self.model = ZhinaoModel(config, linear_method, lora_config=lora_config)...

Did you push the changes?

Also feel free to add to https://github.com/vllm-project/vllm/blob/main/docs/source/models/supported_models.rst and README https://github.com/vllm-project/vllm?tab=readme-ov-file#about

Spec decode tests start failing in main branch after this PR https://buildkite.com/vllm/ci/builds/6784#018f551e-d727-491c-be34-9d9fa29f4ea4

can you reproduce this with just outlines https://github.com/outlines-dev/outlines?tab=readme-ov-file#efficient-json-generation-following-a-json-schema