llm.c icon indicating copy to clipboard operation
llm.c copied to clipboard

Add explicit HuggingFace cache dir

Open gordicaleksa opened this issue 1 year ago • 2 comments

Some of these datasets can be fairly large and I don't like the fact it's hard to figure out where HuggingFace is storing all of it.

I've set it to the default location in the code, but at least it's now explicit.

e.g. I have 2 partitions and HF ends up saving on the one that has only few GBs as opposed to few TBs.

gordicaleksa avatar Jun 20 '24 22:06 gordicaleksa

hmm i feel like this change makes assumptions, and overrides possible pre-existing user configurations of cache dirs

karpathy avatar Jun 20 '24 23:06 karpathy

Can you maybe give me an example? Like if they were hardcoding an env variable?

Because this is precisely what huggingface does behind the scenes, makes ~./.cache the default location and I haven't changed that, just made it explicit.

gordicaleksa avatar Jun 21 '24 08:06 gordicaleksa