Isotr0py
Image input also works now: ### Example Code ```python import torch from vllm import LLM, SamplingParams from vllm.sequence import MultiModalData # images = torch.load("images/stop_sign_pixel_values.pt") images = torch.load("images/cherry_blossom_pixel_values.pt") # for image_prompt...
@simon-mo Can you take a look at this?
@ywang96 @DarkLight1337 I have rebased the code to support the new vision API. Please take a look if you have bandwidth, thanks!
But persimmon model is not a vision model and it doesn't support lora as well currently. So seems that it's no need to add extra model flag for it.
Seems that the failed workflow is related to the network issue...
The change in image size is just to prevent OOM in test. Using `size_factors=1.0` will cause OOM in the test.
> Hmm, I'm concerned that limiting the image size would undermine the completeness of the test. It looks like the OOM also happens on the other models so it might...
> Hmm, I'm concerned that limiting the image size would undermine the completeness of the test. It looks like the OOM also happens on the other models so it might...
Hmmm, I can run the example above on latest commit without any error on my CPU environment: ``` INFO: Started server process [2198956] INFO: Waiting for application startup. INFO: Application...