Slow download from HuggingFace Hub (capped at 10.5 MB/s)

[Andrei-Aksionov]

Bug description

Sometimes the download speed is pretty slow, no more than 10.5 MB/s. For example, gemma-2-2b takes a lot of time to download.

In download.py the code sets hf_transfer variable via constants: https://github.com/Lightning-AI/litgpt/blob/fe96c6366ad8fd20a632673bd6f344bd9b18ca04/litgpt/scripts/download.py#L79-L82

But it doesn't help. Even if set the value through os.environ["HF_HUB_ENABLE_HF_TRANSFER"]="1".

But, if to export the env variable before running a script one can notice a significant speed-up:

export HF_HUB_ENABLE_HF_TRANSFER=1

Most likely, HF Hub checks the env variable upon initialization. The code needs to be fixed.

What operating system are you using?

macOS

LitGPT Version

Version: 0.5.4.dev1

[rasbt]

I think this is fixed now via #1899

[Andrei-Aksionov]

Nope :) Still the same: the downloading speed is capped at 10.5, but if I enable HF_HUB_ENABLE_HF_TRANSFER before downloading it is superfast.

[rasbt]

So weird, I wasn't able to reproduce!

[ahsannawazch]

still the issue is not fixed. I am downloading Qwen2 VL 2b instruct model in lightning.ai studio and takes a lot of time. 3x slower downloading than kaggle.

[Respaired]

it goes beyond these models, it's an HF thing i believe and not Litgpt

[ahsannawazch]

it goes beyond these models, it's an HF thing i believe and not Litgpt

well I tried following and it went upto 500-600 MB/s. Took few seconds to download the same model.

1. export HF_HUB_ENABLE_HF_TRANSFER=1
2. pip install hf_transfer