Roger Wang
Roger Wang
Thanks for reporting! I will take a look at this issue.
Hey I just want to let you know this is still on my list of TODO but I simply had limited bandwidth with other priorities. Sorry for the inconvenience!
@njhill Can we close this PR?
@youkaichao I think you might have the most context in this?
Sorry for the delay - I was busy with Pixtral release last week but will review this PR this week!
@cooleel We decided to work on adding prefix caching for multimodal models on V1 instead since there are some fundamental changes on how cache manager is designed. Stay tuned and...
Thank you for this RFC - imo we should support OpenAI compatibility at our best effort so working on `finished_reason` aligns with our goal. I do think there's a value...
> > Is there a roadmap for 'openai vllm server supports video interface' feature? Thank you > > Thank you. It is a very critical feature, but it needs a...
> @ywang96 I am quite interested in using `vllm` for high-performance video captioning. This will be tremendously helpful for furthering research on video generation from language. > > I was...
> > AFAIK, only Llava-OneVision supports multi-video captioning once #8905 is merged, and this is more of a model capability than of the inference infrastructure capability > > Agree. Thanks...