Jiaxin Shan

Gateway returns not meaningful response when pod is running but container not ready

2

### 🐛 Describe the bug We made few changes in recent weeks to make sure response is explainable. I still see some case not expected today. ``` Pod Running -...

## Pull Request Description fix env error caused by helm set command, use —set-string for env bool. It will keep inject the probs which cause failures for large models. ##...

## Pull Request Description It helps to only consider the engine pod for multi-node inference. Ray worker doesn’t have http server running and it can not expose any application but...

### 🐛 Describe the bug   I am not sure that's all related to timeout setting. https://github.com/vllm-project/aibrix/pull/879 ### Steps to Reproduce send > 20k prompts to the server with...

Installation: Separate CRDs from manifest installation

1

### 🚀 Feature Description and Motivation Follow up https://github.com/vllm-project/aibrix/pull/793. Previously, we use `apply --server-side` as a temporary solution but `field ownership` mechanism will check field owner, if user mix use...

[RFC] Support different inference engines like vLLM, SGLang, TensorRT-LLM

2

### Summary We aim to enhance the serving control plane by supporting multiple inference engines, including vLLM, SGLang, and TensorRT-LLM. These engines have distinct performance optimizations and use cases, and...

### 🚀 Feature Description and Motivation In DeepSeek-R1 examples, we notice setting up ReadWriteMany volume is kind of painful for cloud agnostic environments. S3FS based solutions are not very efficient....

OSError: /models/deepseek-r1 does not appear to have a file named configuration_deepseek.py. Checkout 'https://huggingface.co//models/deepseek-r1/tree/None' for available files.

8

### 🐛 Describe the bug ``` INFO 03-09 00:16:28 api_server.py:913] args: Namespace(subparser='serve', model_tag='/models/deepseek-r1', config='', host=None, port=8000, uvicorn_log_level='warning', allow_credentials=False, allowed_origins=['*'], allowed_methods=['*'], allowed_headers=['*'], api_key=None, lora_modules=None, prompt_adapters=None, chat_template=None, chat_template_content_format='auto', response_role='assistant', ssl_keyfile=None, ssl_certfile=None, ssl_ca_certs=None,...

### 🚀 Feature Description and Motivation I received feedback that users are encountering issues where the `aibrix-controller-manager` cannot list resources of type "mutatingwebhookconfiguration" from the master branch. After following up...

### 🚀 Feature Description and Motivation We encounter some stability issues in the past, https://github.com/vllm-project/aibrix/pull/825 was proposed to address such issues. We should use this opportunity to further check current...