aibrix

aibrix copied to clipboard

## Pull Request Description Alway -> Always --- Contribution Guidelines (Expand for Details) We appreciate your contribution to aibrix! To ensure a smooth review process and maintain high code quality,...

eltociear

[CICD] Release docker images on GHCR

1

### 🚀 Feature Description and Motivation Since Docker Hub has pull limits, switching to using GitHub Container Registry might be better for releasing our images. It could save us some...

gaocegege

[Ecosystem] NIM inference with AIBrix

1

### 🚀 Feature Description and Motivation I’m not entirely certain if this aligns with our goals. Some potential users are looking to integrate NIM (NVIDIA Inference Manager), which is already...

gaocegege

[Question] How to access the vLLM-Vineyard integration code mentioned in Distributed KV Cache documentation?

5

The [Distributed KV Cache documentation](https://aibrix.readthedocs.io/latest/features/distributed-kv-cache.html) references a customized vLLM implementation with Vineyard integration. However, I couldn't locate the corresponding code implementation. Could you help me clarify: 1. Is the customized...

cheyang

v0.3.0 roadmap

9

### 🚀 Feature Description and Motivation I create this issue to track the v0.3.0 items we like to work on. We actually have a milestone https://github.com/aibrix/aibrix/milestone/9 to track all issues...

Jeffwan

[router] Document supported APIs

6

### 🚀 Feature Description and Motivation We should document the supported APIs. Besides this, I wanna ask if embedding APIs are supported. ### Use Case N/A ### Proposed Solution _No...

gaocegege

[RFC] Add model API as the umbrella for model centric deployment

3

### 🚀 Feature Description and Motivation As we discussed offline, we consider to provide a Model API object, designed to simplify and unify the deployment of models by managing all...

Jeffwan

## Pull Request Description Made it more thread safe especially regarding accessing TreeNode data structure. Made variables of TreeNode private and made all them accessed through Getter functions. **Important: Before...

gangmuk

Enable `apply` to replace `create` in manifest deployment

8

### 🚀 Feature Description and Motivation create is very tricky and not good for version upgrade. We get chance to lose the fixed ip. Let's see if there's any options...

Jeffwan

Research & Industry Collaboration Invitation

4

### 🚀 Feature Description and Motivation We're looking for contributors and collaborators to join efforts in pushing forward AI infrastructure research and industry adoption. If you're interested in LLM infrastructure,...

Jeffwan