aibrix
aibrix copied to clipboard
Support direct download of models from Hugging Face for deployment
https://github.com/vllm-project/aibrix/blob/main/samples/kvcache/l1cache/vllm.yaml
It seems that models need to be downloaded from storage to local pods. Can we directly download models from Hugging Face to pods like how we deploy large models with vLLM? This would be simpler and more convenient.