text-generation-inference

text-generation-inference copied to clipboard

The Falcon 40B model occupies nearly 90G of space, takes a long time to download, and does not know the download progress. Is there any other way to download the model first and load it into TGI?

text-generation-server download-weights MODEL_ID is probably what you're looking for.

You can also add the progressbar back using the env variable HF_HUB_ENABLE_HF_TRANSFER=0 (but download will be slower)

Hi @Narsil

Thanks for the support.

Edit: I figure out :)

Perfect. Closing this issue then.