paolovic
paolovic
Hi all, does Ray 2.44.1 support VLLM V1? `vLLM==0.8.1` For some reason my deployment is stuck...it loads forever even with `VLLM_WORKER_MULTIPROC_METHOD="spawn"` this is my `deployments.yaml` that I spawn with `serve...
> [@paolovic](https://github.com/paolovic) could you try vllm==0.8.4? 0.8.1 is known to have some bugs, and 0.8.4 fixed some security issue yes, unfortunately, it doesn't help
> Hi [@paolovic](https://github.com/paolovic) , can you post the logs to show which step you got stuck? Please also show the ray version you used. true. I will update my post,...
So, actually, it was weird cascade of dependency mismatches, hidden by `pip==25.1`, with `pip==24.0` we were able to track the root cause down. In the end, it had nothing to...
did you set `max_new_tokens` >0?
Hi @marcelodiaz558, hi @Endebert, can you verify if this fits your needs? Thank you and best regards, Paul
Hi @PedroF43 , if this is related to this PR, could you provide an example, please? Thank you and best regards, Paul
> > Hi @PedroF43 , if this is related to this PR, could you provide an example, please? > > Thank you and best regards, Paul > > this PR...
Hi @K-Mistele could you have a look, please?
Hi @Endebert , I got a notification from you about a potential bug containing two JSONs as `arguments`. Did you delete this comment because it is resolved?