Roger Wang

[RFC]: Multi-modality Support on vLLM

98

[[Open issues - help wanted!]](https://github.com/vllm-project/vllm/issues/4194#issuecomment-2102487467) **Update [11/18] - In the upcoming months, we will focus on performance optimization for multimodal models as part of vLLM V1 engine re-arch effort** **P0**...

[Model] Initial Support for Chameleon

7

This PR kicks off the initiative to add support for Chameleon - Mixed-Modal Early-Fusion Foundation Models from Meta AI. Currently its goal is to match the `transformers` capability to generate...

[Core][VLM] Support image embeddings as input

10

This PR adds the support for passing image embeddings as input so that they can be directly consumed by the language model. Example usage ``` python # Refer to the...

[Hotfix][VLM] Fixing max position embeddings for Pixtral

1

Previously, max position embeddings was missing from the config and thus set to 8192 by default, causing generation issue when current context window is over 8192. This PR hotfixes this...

[Feature]: Consolidate performance benchmark datasets

2

### 🚀 The feature, motivation and pitch On vLLM we have two main benchmark scripts ([benchmark_throughput.py](https://github.com/vllm-project/vllm/blob/main/benchmarks/benchmark_throughput.py) and [benchmark_serving.py](https://github.com/vllm-project/vllm/blob/main/benchmarks/benchmark_serving.py)) to measure the performance of vLLM. However, the dataset sampling functions are...

[Usage]: Clean up Engine Args & Documentation

1

### Your current environment Currently vLLM has a lot of engine arguments listed here https://docs.vllm.ai/en/latest/serving/engine_args.html. Over time as we add more and more features to vLLM, this list will be...

[V1][Core] Fix memory issue with logits & sampling

6

Reopened from accidentally merged #13721 This PR is a followup to https://github.com/vllm-project/vllm/pull/13594#issuecomment-2676358868 that describes the memory issue during online serving even after sampler profiling is added to `profile_run`. After some...

fix(crud-web-apps): fix condition checking when missing message

2

In the current code of `get_status_from_conditions` it is assumed that "reason" and "message" will always co-exist in a `condition` https://github.com/kubeflow/kubeflow/blob/78969ee8246a6f11295378fc61273b1f2dbe07a8/components/crud-web-apps/jupyter/backend/apps/common/status.py#L160-L163 This is in fact a wrong assumption and thus a...

[DNM][CI] Test H100 on CI

1

PLEASE FILL IN THE PR DESCRIPTION HERE ENSURING ALL CHECKLIST ITEMS (AT THE BOTTOM) HAVE BEEN CONSIDERED. ## Purpose Example buildkite setup to use H100 on CI ## Test Plan...

PLEASE FILL IN THE PR DESCRIPTION HERE ENSURING ALL CHECKLIST ITEMS (AT THE BOTTOM) HAVE BEEN CONSIDERED. ## Purpose ## Test Plan ## Test Result --- Essential Elements of an...