Jiaxin Shan
Jiaxin Shan
### 🐛 Describe the bug  ### Steps to Reproduce N/A ### Expected behavior N/A ### Environment N/A
### 🐛 Describe the bug  ``` Warning FailedMount 25s (x19 over 37m) kubelet (combined from similar events): MountVolume.SetUp failed for volume "kube-api-access-wfmh2" : write /var/lib/kubelet/pods/26af5e70-f9b9-4c12-a9de-dcf2d4e93689/volumes/kubernetes.io~projected/kube-api-access-wfmh2/..2025_02_18_00_11_37.99831085/namespace: no space left on...
### 🚀 Feature Description and Motivation Copy some TODO items from issue https://github.com/aibrix/aibrix/pull/650 WIP items: - [ ] Improving aibrix/benchmarks/generator/client.py with async and streaming mode - [ ] TTFT -...
### 🚀 Feature Description and Motivation This issue is found by @gangmuk Technically, 1. the gateway router fetches the vLLM pods every 50ms, and calculate the running/pending/swapped request and make...
### 🐛 Describe the bug  ### Steps to Reproduce _No response_ ### Expected behavior _No response_ ### Environment _No response_
### 🚀 Feature Description and Motivation We have some initial work here. https://github.com/aibrix/aibrix/tree/main/benchmarks in v0.1.0 testing. however, these scripts are not polished very well. Since we did lots of testing...
### 🚀 Feature Description and Motivation Preble (https://arxiv.org/abs/2407.00023) did solid work on prefix-cache and load-aware routing. The prefix-cache aware version we are implementing is a little bit different from Preble,...
### 🚀 Feature Description and Motivation In the past, we use volcano engine as the primary platform to test aibrix. Now, it's time to test against other public cloud providers....
### 🚀 Feature Description and Motivation There is few cases for migrating the grpc-ext-proc server to a Python code base. This change is driven by two main factors that would...
### 🐛 Describe the bug  ### Steps to Reproduce deploy the models ``` vllm serve Qwen/Qwen2.5-Coder-7B-Instruct --enable-lora --lora-modules model-1=VERSIL91/10627788-942b-4b44-b5f5-167c4b543f2c model-2=VERSIL91/10627788-942b-4b44-b5f5-167c4b543f2c model-3=VERSIL91/10627788-942b-4b44-b5f5-167c4b543f2c model-4=VERSIL91/10627788-942b-4b44-b5f5-167c4b543f2c --max-lora-rank 64 ``` send the request ```...