Ensure aibrix_kvcache Torch version stays compatible with latest vLLM/SGLang releases

[Jeffwan]

🚀 Feature Description and Motivation

We’ve hit issues where the aibrix_kvcache Python package cannot be installed or used together with the latest vLLM/SGLang versions due to Torch version incompatibilities.

Right now, the Torch version we build aibrix_kvcache against is not guaranteed to match (or be compatible with) the Torch versions required by recent vLLM/SGLang releases. This makes it hard for users to simply run:

pip install aibrix_kvcache

then user has to build from scratch which is inconvenient.

Use Case

N/A

Proposed Solution

No response

[Jeffwan]

seems the key problem is the sglang and vllm torch compatibility is not always aligned.