Simon Mo comments

Results 313 comments of


                                            Simon Mo

[CI] Make mistral tests pass

I don't think it the file is included in the test yaml.

[Frontend] add tok/s speed metric to llm class when using tqdm

This is erroring in CI https://buildkite.com/vllm/ci/builds/6751#018f529b-4a51-431a-b1d0-0a4daeb8d49f/319-360 ``` 35%|███▌ | 7/20 [00:04

[Frontend] add tok/s speed metric to llm class when using tqdm

``` Processed prompts: 100%\|██████████\| 1000/1000 [01:02

[Usage]: doubt on computational complexity

For a string prompt, we do not hash it incrementally. For example, (for illustrative purpose), string ABCD is hashed: `hash("A"), hash("AB"), hash("ABC"), hash("ABCD")` to produce 4 hash values and used...

[Bug]: Loading GenerationConfig to SamplingParams.stop_token_ids interfere with ignore_eos=True

Would this PR fixes it https://github.com/vllm-project/vllm/pull/4468

chunked-prefill-doc-syntax

@rkooo567 plz push

Bump pydantic version from 1.10.13 to >= 1.9.0, < 3

Thank you for your contribution. I will close this in favor of #2531

[Bugfix] add truncate_prompt_tokens to work offline, directly from LLM class.

@tdoublep can you help review this?

[Frontend] Support complex message content for chat completions endpoint

nice! i don't even though this schema existed in the first place

Adding `/get_tokenizer` to api_server for lm-evaluation-harness ease integration.

In these cases > models/tokenizer can be private, or lm-evaluation-harness and vllm can be in the same network but without internet access. wouldn't the user of lm-evaluation-harness be able to...