SYing
SYing
> [@ying2025](https://github.com/ying2025) if you do not use specific features like lora, you can easily replace vLLM image with SGLang image. AIBrix should be compatible with it. If you have some...
When executing this command, the $KUBERAY_GEN_RAY_START_CMD blocks the process, preventing the subsequent execution of the vllm serve command: vllm serve /models/deepseek-r1 --trust-remote-code --served-model-name deepseek-r1-671b --tensor-parallel-size 16 --distributed-executor-backend ray --uvicorn-log-level warning"；so,...
ray, version 2.42.0
> 1. Did you add `block: 'false'` in the `rayStartParams`? this is required to remove the `--block` in startup command > 2. Underneath operator kuberay has a bug on disabling...
> 2\. v1.2.1-patch Will this code change be merged into ray, or can you provide the relevant code?
> [@ying2025](https://github.com/ying2025) Yeah, it will be part of kuberay. I am asking one engineer to help with it. here's the code branch [ray-project/kuberay@91e1c26#diff-cc9abb27aaceca3f10193e2ab35fb00dca44b8858709c5c0f4df751c1387291aR576](https://github.com/ray-project/kuberay/commit/91e1c26fbf1fc0f505ff7d16b70cf8228ed62ec4#diff-cc9abb27aaceca3f10193e2ab35fb00dca44b8858709c5c0f4df751c1387291aR576) and original issue [#245 (comment)](https://github.com/vllm-project/aibrix/issues/245#issuecomment-2394811082) ok, thanks
@varungup90 I meet the same problem, prefill-0 pod is normal, prefill-1 is 404.
> Can you describe the DO and D1, and share the label key/values. I want to check the value of pod-group-index label. @varungup90 D0: D1: P0: P1:
> pod-group-index ok, I'll try updating it. I just pulled the image a few days ago.
> vllm need ray cluster to cross-node, we need to launch ray inside the vllm to use PodGroupSize. are you able to use sglang at this moment? yes, I can...