aibrix
aibrix copied to clipboard
Consider to integrate the LLM evaluation metrics to Autoscaling object
🚀 Feature Description and Motivation
We already define a few autoscaling evaluation metrics like provision efficiency, SLO violations, resource usage etc.
If would be great for controller to evaluate it's autoscaling performance.
Use Case
Help user understand how autoscaling are performed.
Proposed Solution
Enable the evaluation or monitoring check and collect feedbacks in HPA object
This would be long term innovation and we do not have time to work on it now. Leave it to v0.2.0 or even future releases