Jiaxin Shan
Jiaxin Shan
/cc @Yard1 @simon-mo Please help review the change
@jeejeelee @simon-mo @njhill Please help review this change. It is mainly used to expose the right `root` and `parent` information in the model card.
@simon-mo @jeejeelee Could you please review this change when you get a chance? Both `serving_xx` and `api_server.py` are frequently updated, and I've had to rebase several times to resolve conflicts....
@simon-mo suggestion accepted and please take another look
/cc @DwyaneShi
@emeraldbay Thanks for reaching out. We have not kicked off the integration work yet. We planned to support some request migration flow but do not have bandwidth at this moment....
@emeraldbay it really depends on the requirements from users. if there're enough interest or suggestion from user side, we will prioritize the work. Seems you expect some general interface which...
@kerthcet I agree that after v0.2.0, we will have a solid baseline of features, and ensuring production-grade quality should be our top priority. We can discuss this further and align...
@gaocegege @kerthcet We do see lots of users do not like ray in distributed serving due to the its overhead and debug-ability. Supporting cloud native way to run vLLM in...
@ying2025 multiple cluster support would be in future release, along with other cloud GPU features. probably in v0.5.0. If you have urgent requirements, feel free to let me know.