aibrix
aibrix copied to clipboard
vllm-project
Enhance parallel download from HuggingFace
🚀 Feature Description and Motivation
The current implementation of multi threading for downloading model from the HuggingFace is directly using
max_workers in snapshot_download.
1 thread = 1 file download
It doesn't do parallel downloading one file
Use Case
No response
Proposed Solution
No response
I move to v0.2.0 for now. Let's concentrate more on the stability of the entire solution and then come back to such feature.