`No such file or directory` when setting `cache_dir`

[rcalland]

System Info

• transformers version: 4.26.1
• Platform: macOS-13.1-arm64-arm-64bit
• Python version: 3.10.9
• Huggingface_hub version: 0.13.1
• PyTorch version (GPU?): not installed (NA)
• Tensorflow version (GPU?): not installed (NA)
• Flax version (CPU?/GPU?/TPU?): not installed (NA)
• Jax version: not installed
• JaxLib version: not installed
• Using GPU in script?: no
• Using distributed or parallel set-up in script?: no

Who can help?

No response

Information

• [ ] The official example scripts
• [ ] My own modified scripts

Tasks

• [ ] An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• [ ] My own task or dataset (give details below)

Reproduction

Note that cache_model directory exists:

from transformers import CLIPProcessor, CLIPModel

processor = CLIPProcessor.from_pretrained("openai/clip-vit-base-patch32", cache_dir="cache_model")


inputs = processor(text=["a photo of a cat", "a photo of a dog"], images=None, return_tensors="np", padding=True)

print(inputs)

Expected behavior

in recent versions of transformers, the code snippet above produces the following error:

FileNotFoundError: [Errno 2] No such file or directory: 'cache_model/models--openai--clip-vit-base-patch32/snapshots/e6a30b603a447e251fdaca1c3056b2a16cdfebeb/preprocessor_config.json'

whereas with transformers==4.20.1, the snippet successfully runs

[sgugger]

I just tried your code sample and it runs without issue on both main and 4.26.1. Are you sure "cache_model" is in a folder you have write access to?

[rcalland]

hi @sgugger, you are correct, I just made a fresh virtualenv and my snippet indeed does work. However, it doesn't work in my dev environment, nor in my projects CI, so I suspect theres an issue with a dependency somewhere?

I dug into what is happening inside cache_model in both environments, and found a clue:

In my new virtualenv:

lrwxr-xr-x  1 richard  staff   136B Mar 14 14:38 preprocessor_config.json -> /Users/richard/Projects/huggingface_bug/cache_model/models--openai--clip-vit-base-patch32/blobs/5a12a1eb250987a4eee0e3e7d7338c4b22724be1

in my dev environment:

lrwxr-xr-x  1 richard  staff    96B Mar 14 15:05 preprocessor_config.json -> cache_model/models--openai--clip-vit-base-patch32/blobs/5a12a1eb250987a4eee0e3e7d7338c4b22724be1

you can see that the former creates a symlink to an absolute path, whereas the latter uses a relative path, which may explain why it cannot find the actual file.

Do you have any ideas what could be causing this behaviour?

[sgugger]

Might be a different version of huggingface_hub?

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.