aibrix
aibrix copied to clipboard
Support multimodal models
🚀 Feature Description and Motivation
Support the deployment and accelerated inference of multimodal models on Aibrix.
Use Case
- Multimodal models are deployed successfully with Aibrix.
- Aibrix can accelerate the inference of multimodal models.
Proposed Solution
No response
Thanks for reporting the issue. We also receive several feedback on the kvcache offloading acceleration and lora for Text2Image cases. We will consider few cases to accelerate multimodal cases.