[Question]: Why set "cache_dir" to "/tmp/cache" on macOS when passing mps as device_map?

Open danny-su opened this issue 1 year ago • 3 comments

Describe the issue

Why set "cache_dir" to "/tmp/cache" on macOS when passing mps as device? This setting creates double caches. Most of my applications use $HOME/.cache/huggingface/pub as the cache directory.

Feb 29 '24 10:02 danny-su

Hi @danny-su, this design is intended for Linux machines. We plan to remove it soon.

Mar 01 '24 09:03 iofu728

@iofu728 I want it to run on macOS. It seems that PyTorch doesn't support BF16 on macOS for now, even though macOS Sonoma has supported BF16. Therefore, it is better to allow the users to pass in "mps" as a device and torch.float32 as torch_dtype.

Mar 01 '24 14:03 danny-su

Hi @danny-su, thank you for your suggestion. We will add a custom torch_dtype.

Mar 04 '24 08:03 iofu728