ray
ray copied to clipboard
[HPU] [Serve] [experimental] Add vllm HPU support in vllm example
Why are these changes needed?
This PR adds vllm HPU support in vllm example (https://github.com/ray-project/ray/pull/45430). The added codes will check whether the HPU device exists before allocating resources to vllm actors. If it exists, HPU resources are used, otherwise GPU resources are still used.
Related issue number
Checks
- [ ] I've signed off every commit(by using the -s flag, i.e.,
git commit -s
) in this PR. - [ ] I've run
scripts/format.sh
to lint the changes in this PR. - [ ] I've included any doc changes needed for https://docs.ray.io/en/master/.
- [ ] I've added any new APIs to the API Reference. For example, if I added a
method in Tune, I've added it in
doc/source/tune/api/
under the corresponding.rst
file.
- [ ] I've added any new APIs to the API Reference. For example, if I added a
method in Tune, I've added it in
- [ ] I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
- Testing Strategy
- [ ] Unit tests
- [ ] Release tests
- [ ] This PR is not tested :(