Cyrus Leung

Just a heads-up that #4355 uses the official type definitions from the `openai` Python library. This ensures consistency with using `openai.Client` to access the server. I think there is no...

> I have rebased and adapted the code, making this one much smaller, thanks @DarkLight1337 👍 > > @simon-mo This is ready for another round I'd say. The Neuron test...

Can the same tokenizer be used to apply the chat template as well?

The LLaVA test passes on my end (with both outputs matching the HF output shown in CI). Does anyone have a clue what might cause it to fail in CI?...

I think #3512 should make this easier by using the same tokenizer for `LLMEngine` and `OpenAIServing`.

@ywang96 Any thoughts about this?

I managed to get the `entrypoints` test to run in a single command. However, I now get this warning when running `test_oot_registration_for_api_server` after changing `multiprocessing` to use the `'spawn'` [start...

#3734 caused quite a few merge conflicts. Hopefully I didn't break anything.

I'm not that familiar with how the nccl library is handled in vLLM. How would using relative imports cause the library to not be loaded?

I think it should be OK now.