aibrix icon indicating copy to clipboard operation
aibrix copied to clipboard

Support direct download of models from Hugging Face for deployment

Open libin817927 opened this issue 7 months ago • 0 comments

https://github.com/vllm-project/aibrix/blob/main/samples/kvcache/l1cache/vllm.yaml

It seems that models need to be downloaded from storage to local pods. Can we directly download models from Hugging Face to pods like how we deploy large models with vLLM? This would be simpler and more convenient.

libin817927 avatar May 29 '25 06:05 libin817927