sd-scripts icon indicating copy to clipboard operation
sd-scripts copied to clipboard

How to change HuggingFace directory

Open mbastias opened this issue 1 year ago • 10 comments

I'm sorry to use an issue for this. My C: drive is very full and the hugginface cache is using about 30Gb so I want to move it to one of my bigger drives. On SD webui I managed to do it by doing a set for the HF_HOME, I thought that this setting was going to carry over to Kohya but it did not, as it downloaded the cached files again.

Is it possible to change the directory? And if it is, how?

mbastias avatar Apr 06 '23 15:04 mbastias

I'm not sure about caching, but according to the next issue. setting environmental variable HF_HOME will solve it: https://github.com/huggingface/transformers/issues/8703

kohya-ss avatar Apr 10 '23 12:04 kohya-ss

Where do I set it? on the venv? Sorry I'm very new this environment.

mbastias avatar Apr 11 '23 17:04 mbastias

@mbastias if you are running bash, which most Linux environments use by default, in your shell:

export HF_HOME=${HOME}/cache

A better option is to add this to the default startup. To do this, add export HF_HOME=${HOME}/cache to the ${HOME}/.profile file.

For more detail, reference the .profile man page.

sdake avatar May 17 '23 03:05 sdake

I'm on Windows and I have the same problem. My C drive is full and I would like to move the .cache/huggingface folder to another drive.

rezponze avatar May 18 '23 21:05 rezponze

I'm on Windows and I have the same problem. My C drive is full and I would like to move the .cache/huggingface folder to another drive.

I seem to have solve it adding "set HF_HOME=D:\SDCache" like this:

@echo off

:: Activate the virtual environment
call .\venv\Scripts\activate.bat

:: Validate the requirements and store the exit code
python.exe .\tools\validate_requirements.py

set HF_HOME=D:\SDCache

:: If the exit code is 0, run the kohya_gui.py script with the command-line arguments
if %errorlevel% equ 0 (
    python.exe kohya_gui.py %*
)

Replace D:\SDCache with the path you want

mbastias avatar May 19 '23 04:05 mbastias

I'm on Windows and I have the same problem. My C drive is full and I would like to move the .cache/huggingface folder to another drive.

I seem to have solve it adding "set HF_HOME=D:\SDCache" like this:

@echo off

:: Activate the virtual environment
call .\venv\Scripts\activate.bat

:: Validate the requirements and store the exit code
python.exe .\tools\validate_requirements.py

set HF_HOME=D:\SDCache

:: If the exit code is 0, run the kohya_gui.py script with the command-line arguments
if %errorlevel% equ 0 (
    python.exe kohya_gui.py %*
)

Replace D:\SDCache with the path you want

You are the best! Thank you! I wrote the same line in setup.bat to install the acceleration files in a different folder. It really works. And hopefully it won't break anything, huh:)

aearone avatar Jun 12 '23 20:06 aearone

nice work @aearone !

Thank you for sharing your work, -steve

sdake avatar Jun 15 '23 06:06 sdake

rather thank you @mbastias for solving this problem for Windows.

thank you, -steve

sdake avatar Jun 15 '23 06:06 sdake

that seems hard. i just type env down in the search box in windows 11. and that opens the environmental variables. i click new. and add this variable name HF_HOME variable value H:\Cache then okay. and it seems to be persistent.

mylesgoose avatar Mar 17 '24 03:03 mylesgoose

$Env:HF_HOME = "F:\HF_Cache" 将 F:\HF_Cache 替换为您想要的路径 powershell里面执行

SinoCHU avatar Mar 30 '24 12:03 SinoCHU