requests.exceptions.ConnectionError

[taoari]

System Info

model=gpt2 volume=$HOME/.cache/huggingface/hub num_shard=1 docker run --gpus all --shm-size 1g -p 8081:80 -v $volume:/data ghcr.io/huggingface/text-generation-inference:0.8.1 --model-id $model --num-shard $num_shard

Information

• [X] Docker
• [ ] The CLI directly

Tasks

• [X] An officially supported command
• [ ] My own modifications

Reproduction

It is very strange that 1 out of 6 or 7 times it works. But most of times I got the following error:

requests.exceptions.ConnectionError: HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /api/models/gpt2 (Caused by NameResolutionError("<urllib3.connection.HTTPSConnection object at 0x7fc5b7a2efa0>: Failed to resolve 'huggingface.co' ([Errno -3] Temporary failure in name resolution)"))

Expected behavior

Should always work

[Narsil]

Your DNS resolver is the issue here. Most likely linked to your cluster not appreciating the throughput we're sending it.

You could use -e HF_HUB_ENABLE_HF_TRANSFER=0 to reduce the bandwidth/network calls or figure out what's happening to the DNS server.

[poojitharamachandra]

-e HF_HUB_ENABLE_HF_TRANSFER=0 is not recognized

error: unexpected argument 'HF_HUB_ENABLE_HF_TRANSFER=0' found

Usage: text-generation-launcher [OPTIONS]

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.