Jiaxin Shan
Jiaxin Shan
The KVCache API Spec still need more tuning, thanks for @vie-serendipity's feedback here https://github.com/vllm-project/aibrix/pull/1055#discussion_r2080872612 I will not close this issue until we get enough feedback from users. From longterm perspective,...
BTW, I think stormservice now can plays the role to support arbitrary role orchstration. that would be a great replacement for current API. We will explore this path and consider...
Seem this is multi_modal model? I do seem some related issues. Can someone confirm whether the multi_modal + prefix cache is supported or not?
I will close this story. If you find it's still a problem, free free to reopen the issue. thanks for @googs1025's fix
I was busy on the open source stuff yesterday. I will spend some time on the review todya
doc build failure is expected, I switch to free version and it only supports public repo. It will come back soon
@kerthcet #740 and #741 have been merged. We can revisit this one now
the doc build is broken after the change 
the ghcr.io part failed as well 
 Seems the rulesets needs to be confirmed by admin