huggingface_hub icon indicating copy to clipboard operation
huggingface_hub copied to clipboard

Process hung when downloading model

Open kiranchitturi opened this issue 6 months ago • 0 comments

Describe the bug

HuggingFace is using TQDM and it is causing the process to hang indefinitely. Are there any library conflicts that could be causing this? How can I get around this error?

Reproduction

No response

Logs

{"event": "PyTorch version 2.1.2 available.", "logger": "datasets", "timestamp": "2024-08-23T00:43:27.482936Z", "level": "info"}
Thread 0x0000000304556000 (most recent call first):
  File "/Users/kiranchitturi/opt/miniconda3/envs/conda_env2/lib/python3.10/threading.py", line 324 in wait
  File "/Users/kiranchitturi/opt/miniconda3/envs/conda_env2/lib/python3.10/threading.py", line 600 in wait
  File "/Users/kiranchitturi/opt/miniconda3/envs/conda_env2/lib/python3.10/threading.py", line 1356 in run
  File "/Users/kiranchitturi/opt/miniconda3/envs/conda_env2/lib/python3.10/threading.py", line 1009 in _bootstrap_inner
  File "/Users/kiranchitturi/opt/miniconda3/envs/conda_env2/lib/python3.10/threading.py", line 966 in _bootstrap

Current thread 0x0000000200933240 (most recent call first):
  File "/Users/kiranchitturi/opt/miniconda3/envs/conda_env2/lib/python3.10/multiprocessing/util.py", line 452 in spawnv_passfds
  File "/Users/kiranchitturi/opt/miniconda3/envs/conda_env2/lib/python3.10/multiprocessing/resource_tracker.py", line 129 in ensure_running
  File "/Users/kiranchitturi/opt/miniconda3/envs/conda_env2/lib/python3.10/multiprocessing/resource_tracker.py", line 162 in _send
  File "/Users/kiranchitturi/opt/miniconda3/envs/conda_env2/lib/python3.10/multiprocessing/resource_tracker.py", line 155 in register
  File "/Users/kiranchitturi/opt/miniconda3/envs/conda_env2/lib/python3.10/multiprocessing/synchronize.py", line 80 in __init__
  File "/Users/kiranchitturi/opt/miniconda3/envs/conda_env2/lib/python3.10/multiprocessing/synchronize.py", line 187 in __init__
  File "/Users/kiranchitturi/opt/miniconda3/envs/conda_env2/lib/python3.10/multiprocessing/context.py", line 73 in RLock
  File "/Users/kiranchitturi/opt/miniconda3/envs/conda_env2/lib/python3.10/site-packages/tqdm/std.py", line 121 in create_mp_lock
  File "/Users/kiranchitturi/opt/miniconda3/envs/conda_env2/lib/python3.10/site-packages/tqdm/std.py", line 97 in __init__
  File "/Users/kiranchitturi/opt/miniconda3/envs/conda_env2/lib/python3.10/site-packages/tqdm/std.py", line 764 in get_lock
  File "/Users/kiranchitturi/opt/miniconda3/envs/conda_env2/lib/python3.10/site-packages/tqdm/contrib/concurrent.py", line 19 in ensure_lock
  File "/Users/kiranchitturi/opt/miniconda3/envs/conda_env2/lib/python3.10/contextlib.py", line 135 in __enter__
  File "/Users/kiranchitturi/opt/miniconda3/envs/conda_env2/lib/python3.10/site-packages/tqdm/contrib/concurrent.py", line 47 in _executor_map
  File "/Users/kiranchitturi/opt/miniconda3/envs/conda_env2/lib/python3.10/site-packages/tqdm/contrib/concurrent.py", line 69 in thread_map
  File "/Users/kiranchitturi/opt/miniconda3/envs/conda_env2/lib/python3.10/site-packages/huggingface_hub/_snapshot_download.py", line 308 in snapshot_download
  File "/Users/kiranchitturi/opt/miniconda3/envs/conda_env2/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 118 in _inner_fn
  File "/Users/kiranchitturi/opt/miniconda3/envs/conda_env2/lib/python3.10/site-packages/FlagEmbedding/BGE_M3/modeling.py", line 72 in load_model
  File "/Users/kiranchitturi/opt/miniconda3/envs/conda_env2/lib/python3.10/site-packages/FlagEmbedding/BGE_M3/modeling.py", line 40 in __init__
  File "/Users/kiranchitturi/opt/miniconda3/envs/conda_env2/lib/python3.10/site-packages/FlagEmbedding/bge_m3.py", line 36 in __init__
  File "/Users/kiranchitturi/opt/miniconda3/envs/conda_env2/lib/python3.10/site-packages/milvus_model/hybrid/bge_m3.py", line 60 in __init__
  File "/Users/kiranchitturi/opt/miniconda3/envs/conda_env2/lib/python3.10/site-packages/milvus_model/hybrid/__init__.py", line 9 in BGEM3EmbeddingFunction
  File "/Users/kiranchitturi/dev/git/PROJECT_DIR/app/pages/script.py", line 31 in get_bge_m3_embedding_function
  File "/Users/kiranchitturi/opt/miniconda3/envs/conda_env2/lib/python3.10/site-packages/streamlit/runtime/caching/cache_utils.py", line 322 in _handle_cache_miss
  File "/Users/kiranchitturi/opt/miniconda3/envs/conda_env2/lib/python3.10/site-packages/streamlit/runtime/caching/cache_utils.py", line 266 in _get_or_create_cached_value
  File "/Users/kiranchitturi/opt/miniconda3/envs/conda_env2/lib/python3.10/site-packages/streamlit/runtime/caching/cache_utils.py", line 239 in __call__
  File "/Users/kiranchitturi/opt/miniconda3/envs/conda_env2/lib/python3.10/site-packages/streamlit/runtime/caching/cache_utils.py", line 210 in wrapper
  File "/Users/kiranchitturi/dev/git/PROJECT_DIR/app/pages/script.py", line 47 in <module>


### System info

```shell
- huggingface_hub version: 0.20.1
- Platform: macOS-10.16-x86_64-i386-64bit
- Python version: 3.10.0
- Running in iPython ?: No
- Running in notebook ?: No
- Running in Google Colab ?: No
- Token path ?: /Users/kiranchitturi/.cache/huggingface/token
- Has saved token ?: False
- Configured git credential helpers: osxkeychain
- FastAI: N/A
- Tensorflow: N/A
- Torch: 2.1.2
- Jinja2: 3.1.2
- Graphviz: N/A
- Pydot: N/A
- Pillow: 10.1.0
- hf_transfer: N/A
- gradio: N/A
- tensorboard: N/A
- numpy: 1.26.2
- pydantic: 2.5.3
- aiohttp: 3.10.5
- ENDPOINT: https://huggingface.co
- HF_HUB_CACHE: /Users/kiranchitturi/.cache/huggingface/hub
- HF_ASSETS_CACHE: /Users/kiranchitturi/.cache/huggingface/assets
- HF_TOKEN_PATH: /Users/kiranchitturi/.cache/huggingface/token
- HF_HUB_OFFLINE: False
- HF_HUB_DISABLE_TELEMETRY: False
- HF_HUB_DISABLE_PROGRESS_BARS: None
- HF_HUB_DISABLE_SYMLINKS_WARNING: False
- HF_HUB_DISABLE_EXPERIMENTAL_WARNING: False
- HF_HUB_DISABLE_IMPLICIT_TOKEN: False
- HF_HUB_ENABLE_HF_TRANSFER: False
- HF_HUB_ETAG_TIMEOUT: 10
- HF_HUB_DOWNLOAD_TIMEOUT: 10

kiranchitturi avatar Aug 23 '24 01:08 kiranchitturi