Alex Brooks
Alex Brooks
## Is your feature request related to a problem? Please describe. In some situations, a user might have multiple devices and multiple models available. At load time, there might be...
This PR finishes exposing multi-image support for Qwen-VL (not Qwen2) as follow-up to https://github.com/vllm-project/vllm/pull/8029. ### Multi-image offline inference example (.generate) ```python from vllm import LLM, SamplingParams from vllm.multimodal.utils import fetch_image...
*This PR is large because it's touching pretty much all of the VLM tests - the place where they are consolidated into, i.e., where new model tests would be added...
Currently, there is some special handling for mllama in the chat utils that result in only one image placeholder being used when multiple images are added. Now that https://github.com/vllm-project/vllm/pull/9095 added...
# What does this PR do? Fixes https://github.com/huggingface/accelerate/issues/2975 This PR adds a `CustomTypesDataLoader`, which allows for the passing of custom iterable types (either under a PyTorch DataLoader, which would normally...
This PR: - adds a patch for supporting IBM's granite vision models - includes the [fix](https://github.com/ggml-org/llama.cpp/pull/11982#event-16394708954) for visual encoders with no CLS, which will also fix other llava models that...
This PR adds support for IBM's granite vision models, which are a variant of LLava Next that use multiple feature layers from the (siglip) visual encoder. It's in draft at...
There are a few checks that vLLM runs when inferring the platform that rely on inspecting the version of the vLLM package (i.e., via `importlib.metadata.version`). If the vLLM package isn't...
This PR fixes the bug outlined in this issue: https://github.com/ggml-org/llama.cpp/issues/10157 As well as discussed in projects leverage llama cpp like ollama: https://github.com/ollama/ollama/issues/7441 https://github.com/ollama/ollama-python/issues/433 ### Summary In `clip.cpp`, we initialize a...
Adds example docs for converting a granite vision model, which is essentially a llava next model with multiple feature layers using siglip for the visual encoder, and a granite language...