aibrix
aibrix copied to clipboard
Support lora metrics in vLLM
🚀 Feature Description and Motivation
FYI
Use Case
No response
Proposed Solution
No response
Problem 1: incorrect label names
{'model_name': 'meta-llama/Llama-2-7b-hf', 'running_lora_adapters': '', 'waiting_lora_adapters': '', 'max_lora': '1'}
_log_gauge_string doesn't need merge actually
https://github.com/vllm-project/vllm/pull/11117 update the code to address it.