Alex Wu

[draft][autoscaler] Node preference plugin

2

## Why are these changes needed? ## Related issue number ## Checks - [ ] I've signed off every commit(by using the -s flag, i.e., `git commit -s`) in this...

## Why are these changes needed? ## Related issue number ## Checks - [ ] I've signed off every commit(by using the -s flag, i.e., `git commit -s`) in this...

Build ARM wheels for Ray

8

Due to user demand, we may eventually want to support ARM builds. For now, here are some instructions on how to build ray from source on ARM. This was tested...

[Frontend] Support OpenAI batch file format

5

This PR adds basic support for the OpenAI batch file format (for chat endpoints). The following help page summarizes the usage options: ``` # python -m vllm.entrypoints.openai.run_batch --help usage: run_batch.py...