Jiaxin Shan
Jiaxin Shan
@ltm920716 multiconnector is the roadmap but I personally didn't get chance to work on it yet. We will come back to you soon on this support. BTW, is this an...
part of #99
this is to avoid over-assignment
@zhangjyr I remember our review feedback earlier is to have decoupled PR from the cache refactoring. cache refacor PR was merged, delay scheduling PR was not out right? I think...
@zhangjyr few questions, - SLO-based routing policy is associated with this issue https://github.com/vllm-project/aibrix/issues/642? - If other routing policies need delay scheduling as a common capability to leverage, is it possible?...
Sounds good. Then this story probably can focus more on how to enable delay scheduling for other routing solution or something bake into default logic that all can leverage ....
@kerthcet #740 is merged. Seems the CI is problematic, it doesn't detect the conflicts now. Please help rebase the change
Sounds good
In v0.3.0, we will just make basic interface compatible and document the current situation. We may move some work to v0.4.0
I notice the error response format is completely incompatible. after checking https://github.com/openai/openai-openapi/tree/master?tab=readme-ov-file - Gateway is missing required fields: type, param - Gateway uses HTTP status code (int) for code instead...