Harry Mellor

icon company @huggingface @vllm-project icon location ML Engineer @huggingface

Results 298 comments of Harry Mellor

Improve configs - `SchedulerConfig`

Failures are relevant, I'll look into it

[CI/Build] migrate static project metadata from setup.py to pyproject.toml

@dtrifiro do you plan to update this PR? It seems like a nice step towards removing `setup.py`

[CI/Build] custom build backend improvements/cleanup

After speaking to @dtrifiro, closing as an updated implementation would require a new PR

Move requirements into their own directory

`requirements-docs.txt` currently lives inside `docs`, so it's not cluttering the root directory like the others. Would you still like `requirements-docs.txt` to live with the rest?

Move requirements into their own directory

Done

Move requirements into their own directory

Good catch @simon-mo, fixed!

Fix missing `kv_caches` and `attn_metadata` in `OpenVINOCausalLM`

I'm not sure how to get the `kv_caches` out of forward context. I tried: ```python flat_kv_caches = [ attn_layer.kv_cache[fwd_ctx.virtual_engine] for attn_layer in fwd_ctx.attn_layers.values() ] ``` but `attn_layers` is an empty...

Fix missing `kv_caches` and `attn_metadata` in `OpenVINOCausalLM`

Ok I managed to run it locally on my machine this time and it should work/be ready for review!

Fix missing `kv_caches` and `attn_metadata` in `OpenVINOCausalLM`

The `HfRunner` is not using vLLM

Fix missing `kv_caches` and `attn_metadata` in `OpenVINOCausalLM`

The test suite does not run on OpenVINO, and that is not going to change in this PR. See https://github.com/vllm-project/vllm/blob/main/.buildkite/run-openvino-test.sh to see the extent of testing on OpenVINO.