Jiaxin Shan
Jiaxin Shan
I feel there're few ways, let's explore them together with `lsof` - 1. create `.tmp` file and do rename once finished - 2. create separate `xxx.lock` and remove it once...
## Problem 1: incorrect label names  {'model_name': 'meta-llama/Llama-2-7b-hf', 'running_lora_adapters': '', 'waiting_lora_adapters': '', 'max_lora': '1'} `_log_gauge_string` doesn't need merge actually
https://github.com/vllm-project/vllm/pull/11117 update the code to address it.
I feel the key is still https://github.com/vllm-project/aibrix/issues/1273, we need to naively support this orchestration type
/cc @happyandslow
/cc @omerap12 here's a related issue. HTTPRoute is not supported for stormservice yet
@xiaolin8 this is not by design, we use aibrix router a lot and doesn't rely on httproute. Yes. I remember there's an issue to track it. If that's an urgent...
@xiaolin8 technically, applicaiton can append the default headers. I got your point. in future release, we will make it as default option to get the router benefits without additional changes,...
this is multi-engine related
/cc @happyandslow