aibrix
aibrix copied to clipboard
Implement cold start manager for lora models
🚀 Feature Description and Motivation
cache locality can be leveraged to reduce model startup time. As user uses up to 128 rank which is kind of large, this feature would be helpful. this is from https://github.com/aibrix/aibrix/issues/22
Use Case
No response
Proposed Solution
No response