Jiaxin Shan

[RFC]: Make API Gateway interface OpenAI compatible

1

### Summary This RFC proposes making the API Gateway interface within AIBrix compatible with OpenAI. We met few issues in the past few days. and @gaocegege also suggest https://github.com/vllm-project/aibrix/issues/732 earlier...

### 🚀 Feature Description and Motivation AIBrix, which is composed of multiple controllers, currently lack of comprehensive monitoring makes it difficult to effectively manage and troubleshoot the system. We at...

### 🐛 Describe the bug This is a follow up issue of https://github.com/vllm-project/aibrix/issues/783 Seems some specific prompt data will fail the test ``` python benchmark_serving.py --backend vllm --model deepseek-ai/deepseek-r1 --trust-remote-code...

RayClusterFleet controllers shows some reconcilation issues

1

### 🐛 Describe the bug  ### Steps to Reproduce use deepseek-r1-local-volume version ### Expected behavior it should create the pods successfully ### Environment commit: 1f96e9a47aab42cf003ed4bd031701eea754332c24b6b10c691ee5a842eeceb1

### 🚀 Feature Description and Motivation We can consider to make code reusable and move to https://github.com/vllm-project/aibrix/tree/main/python/aibrix  This make the generator/benchmark code reusable. ### Use Case As a user,...

v0.4.0 roadmap

3

### 🚀 Feature Description and Motivation We’re actively evolving AIBrix to support more advanced and production-ready LLM serving capabilities. For v0.4.0 and beyond, our roadmap includes: - Prefill & Decode...

upstream connect error or disconnect/reset before headers

1

### 🐛 Describe the bug  Can we list potential reasons for connection issues. In recently benchmark testing, I see a few similar cases. ### Steps to Reproduce Run benchmark,...

[RFC]: Add Support for Prefill/Decode (P/D) Disaggregation in vLLM

1

### Summary To further optimize large-scale LLM inference workloads, we plan to introduce support for Prefill/Decode (P/D) disaggregation in vLLM. This separation allows prefill and decode stages to run on...

[CICD] Optimize kv cache image size

1

### 🚀 Feature Description and Motivation  The image size is super large now, we need to reduce the size a little bit. ``` FROM ubuntu:22.04 RUN apt-get update &&...

### Summary The current KVCache offloading framework is built around assumptions from the vLLM v0 architecture. With the release of vLLM v1, which introduces new cache handling semantics, especially the...