diffusers icon indicating copy to clipboard operation
diffusers copied to clipboard

HFValidationError( '.ggingface_hub.utils._validators.HFValidationError: Repo id must use alphanumeric chars or '-', '_', '.', '--' and '..' are forbidden, '-' and '.' cannot start or end the name, max length is 96: 'CompVis/stable-diffusion-v1-4

Open Renaldas111 opened this issue 2 years ago • 9 comments

Tried to install Dreambooth according to Nerdy Rodent video (https://www.youtube.com/watch?v=w6PTviOCYQY)

I am absolutely new to Linux, and failed on last step running the training. As I understood, something wrong with: export MODEL_NAME="CompVis/stable-diffusion-v1-4" and --pretrained_model_name_or_path=$MODEL_NAME

Traceback (most recent call last): File "/home/renaldas/github/diffusers/examples/dreambooth/train_dreambooth.py", line 657, in main() File "/home/renaldas/github/diffusers/examples/dreambooth/train_dreambooth.py", line 420, in main tokenizer = CLIPTokenizer.from_pretrained( File "/home/renaldas/anaconda3/envs/sd/lib/python3.9/site-packages/transformers/tokenization_utils_base.py", line 1734, in from_pretrained resolved_vocab_files[file_id] = cached_file( File "/home/renaldas/anaconda3/envs/sd/lib/python3.9/site-packages/transformers/utils/hub.py", line 408, in cached_file resolved_file = hf_hub_download( File "/home/renaldas/anaconda3/envs/sd/lib/python3.9/site-packages/huggingface_hub/file_download.py", line 1022, in hf_hub_download cache_dir, repo_folder_name(repo_id=repo_id, repo_type=repo_type) File "/home/renaldas/anaconda3/envs/sd/lib/python3.9/site-packages/huggingface_hub/utils/_validators.py", line 92, in _inner_fn validate_repo_id(arg_value) File "/home/renaldas/anaconda3/envs/sd/lib/python3.9/site-packages/huggingface_hub/utils/_validators.py", line 142, in validate_repo_id raise HFValidationError( '.ggingface_hub.utils.validators.HFValidationError: Repo id must use alphanumeric chars or '-', '', '.', '--' and '..' are forbidden, '-' and '.' cannot start or end the name, max length is 96: 'CompVis/stable-diffusion-v1-4

Renaldas111 avatar Oct 05 '22 15:10 Renaldas111

Hey @Renaldas111,

Could you maybe add a reproducible code snippet here? At the moment, I can only guess what code you have run :sweat_smile:

Thank you :-)

patrickvonplaten avatar Oct 07 '22 13:10 patrickvonplaten

I dont belive its a code issue. I was facing the same issue and was due to the notepadd++ being configured for windows end of line conversion and not unix so was adding hidden characters to the .sh file. Once changed Notepadd to Unix (Edit - EOL - Unix) and saved the file it worked.

IT8055 avatar Oct 08 '22 12:10 IT8055

I had the same problem.

%env MODEL_NAME="CompVis/stable-diffusion-v1-4"
%env INSTANCE_DIR="./input"
%env OUTPUT_DIR="./output"

!accelerate launch train_dreambooth.py \
  --pretrained_model_name_or_path=$MODEL_NAME  \
  --instance_data_dir=$INSTANCE_DIR \
  --output_dir=$OUTPUT_DIR \
  --instance_prompt="a photo of sks kitchen" \
  --resolution=512 \
  --train_batch_size=1 \
  --gradient_accumulation_steps=1 \
  --learning_rate=5e-6 \
  --lr_scheduler="constant" \
  --lr_warmup_steps=0 \
  --max_train_steps=400
env: MODEL_NAME="CompVis/stable-diffusion-v1-4"
env: INSTANCE_DIR="./input"
env: OUTPUT_DIR="./output"
The following values were not passed to `accelerate launch` and had defaults used instead:
	`--num_cpu_threads_per_process` was set to `6` to improve out-of-box performance
To avoid this warning pass in values for each of the problematic parameters or run `accelerate config`.
Traceback (most recent call last):
  File "train_dreambooth.py", line 592, in <module>
    main()
  File "train_dreambooth.py", line 381, in main
    tokenizer = CLIPTokenizer.from_pretrained(args.pretrained_model_name_or_path, subfolder="tokenizer")
  File "/usr/local/lib/python3.7/dist-packages/transformers/tokenization_utils_base.py", line 1748, in from_pretrained
    _commit_hash=commit_hash,
  File "/usr/local/lib/python3.7/dist-packages/transformers/utils/hub.py", line 420, in cached_file
    local_files_only=local_files_only,
  File "/usr/local/lib/python3.7/dist-packages/huggingface_hub/file_download.py", line 1022, in hf_hub_download
    cache_dir, repo_folder_name(repo_id=repo_id, repo_type=repo_type)
  File "/usr/local/lib/python3.7/dist-packages/huggingface_hub/utils/_validators.py", line 92, in _inner_fn
    validate_repo_id(arg_value)
  File "/usr/local/lib/python3.7/dist-packages/huggingface_hub/utils/_validators.py", line 143, in validate_repo_id
    "Repo id must use alphanumeric chars or '-', '_', '.', '--' and '..' are"
huggingface_hub.utils._validators.HFValidationError: Repo id must use alphanumeric chars or '-', '_', '.', '--' and '..' are forbidden, '-' and '.' cannot start or end the name, max length is 96: '"CompVis/stable-diffusion-v1-4"'.
Traceback (most recent call last):
  File "/usr/local/bin/accelerate", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.7/dist-packages/accelerate/commands/accelerate_cli.py", line 43, in main
    args.func(args)
  File "/usr/local/lib/python3.7/dist-packages/accelerate/commands/launch.py", line 910, in launch_command
    simple_launcher(args)
  File "/usr/local/lib/python3.7/dist-packages/accelerate/commands/launch.py", line 400, in simple_launcher
    raise subprocess.CalledProcessError(returncode=process.returncode, cmd=cmd)
subprocess.CalledProcessError: Command '['/usr/bin/python3', 'train_dreambooth.py', '--pretrained_model_name_or_path="CompVis/stable-diffusion-v1-4"', '--instance_data_dir="./input"', '--output_dir="./output"', '--instance_prompt=a photo of sks kitchen', '--resolution=512', '--train_batch_size=1', '--gradient_accumulation_steps=1', '--learning_rate=5e-6', '--lr_scheduler=constant', '--lr_warmup_steps=0', '--max_train_steps=400']' returned non-zero exit status 1.

2feet6inches avatar Oct 17 '22 05:10 2feet6inches

There is no problem if you specify it directly.

%env INSTANCE_DIR="./input"
%env OUTPUT_DIR="./output"

!accelerate launch train_dreambooth.py \
  --pretrained_model_name_or_path="CompVis/stable-diffusion-v1-4"  \
  --instance_data_dir=$INSTANCE_DIR \
  --output_dir=$OUTPUT_DIR \
  --instance_prompt="a photo of sks kitchen" \
  --resolution=512 \
  --train_batch_size=1 \
  --gradient_accumulation_steps=1 \
  --learning_rate=5e-6 \
  --lr_scheduler="constant" \
  --lr_warmup_steps=0 \
  --max_train_steps=400

2feet6inches avatar Oct 17 '22 05:10 2feet6inches

cc @patil-suraj here as well

patrickvonplaten avatar Oct 17 '22 17:10 patrickvonplaten

I couldn't reproduce this, providing model name as env variable works fine for me.

patil-suraj avatar Oct 25 '22 14:10 patil-suraj

I just got that issue .. few hours ago I didn't have any issues running this command:

sudo docker run -it --gpus=all --ipc=host -v $(pwd):/root -e HUGGING_FACE_HUB_TOKEN=$TOKEN smy20011/dreambooth:latest accelerate launch /train_dreambooth.py   --pretrained_model_name_or_path=$MODEL_NAME   --instance_data_dir=$INSTANCE_DIR  --class_data_dir=$CLASS_DIR   --output_dir=$OUTPUT_DIR   --with_prior_preservation --prior_loss_weight=1.0   --instance_prompt="a portrait of sks woman"   --class_prompt="a portrait of woman"   --resolution=512   --center_crop   --train_batch_size=1   --mixed_precision="fp16"   --use_8bit_adam   --gradient_accumulation_steps=1 --gradient_checkpointing   --learning_rate=5e-6   --lr_scheduler="constant"   --lr_warmup_steps=0   --num_class_images=100   --sample_batch_size=4   --max_train_steps=800
The following values were not passed to `accelerate launch` and had defaults used instead:
        `--num_processes` was set to a value of `1`
        `--num_machines` was set to a value of `1`
        `--mixed_precision` was set to a value of `'no'`
        `--num_cpu_threads_per_process` was set to `3` to improve out-of-box performance
To avoid this warning pass in values for each of the problematic parameters or run `accelerate config`.
Traceback (most recent call last):
  File "/opt/conda/lib/python3.7/site-packages/diffusers/configuration_utils.py", line 234, in get_config_dict
    revision=revision,
  File "/opt/conda/lib/python3.7/site-packages/huggingface_hub/file_download.py", line 1022, in hf_hub_download
    cache_dir, repo_folder_name(repo_id=repo_id, repo_type=repo_type)
  File "/opt/conda/lib/python3.7/site-packages/huggingface_hub/utils/_validators.py", line 92, in _inner_fn
    validate_repo_id(arg_value)
  File "/opt/conda/lib/python3.7/site-packages/huggingface_hub/utils/_validators.py", line 143, in validate_repo_id
    "Repo id must use alphanumeric chars or '-', '_', '.', '--' and '..' are"
huggingface_hub.utils._validators.HFValidationError: Repo id must use alphanumeric chars or '-', '_', '.', '--' and '..' are forbidden, '-' and '.' cannot start or end the name, max length is 96: ''.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/train_dreambooth.py", line 695, in <module>
    main()
  File "/train_dreambooth.py", line 376, in main
    args.pretrained_model_name_or_path, torch_dtype=torch_dtype, use_auth_token=True
  File "/opt/conda/lib/python3.7/site-packages/diffusers/pipeline_utils.py", line 372, in from_pretrained
    revision=revision,
  File "/opt/conda/lib/python3.7/site-packages/diffusers/configuration_utils.py", line 261, in get_config_dict
    f"We couldn't connect to '{HUGGINGFACE_CO_RESOLVE_ENDPOINT}' to load this model, couldn't find it"
OSError: We couldn't connect to 'https://huggingface.co' to load this model, couldn't find it in the cached files and it looks like  is not the path to a directory containing a model_index.json file.
Checkout your internet connection or see how to run the library in offline mode at 'https://huggingface.co/docs/diffusers/installation#offline-mode'.
Traceback (most recent call last):
  File "/opt/conda/bin/accelerate", line 8, in <module>
    sys.exit(main())
  File "/opt/conda/lib/python3.7/site-packages/accelerate/commands/accelerate_cli.py", line 43, in main
    args.func(args)
  File "/opt/conda/lib/python3.7/site-packages/accelerate/commands/launch.py", line 837, in launch_command
    simple_launcher(args)
  File "/opt/conda/lib/python3.7/site-packages/accelerate/commands/launch.py", line 354, in simple_launcher
    raise subprocess.CalledProcessError(returncode=process.returncode, cmd=cmd)
subprocess.CalledProcessError: Command '['/opt/conda/bin/python', '/train_dreambooth.py', '--pretrained_model_name_or_path=', '--instance_data_dir=', '--class_data_dir=', '--output_dir=', '--with_prior_preservation', '--prior_loss_weight=1.0', '--instance_prompt=a portrait of sks woman', '--class_prompt=a portrait of woman', '--resolution=512', '--center_crop', '--train_batch_size=1', '--mixed_precision=fp16', '--use_8bit_adam', '--gradient_accumulation_steps=1', '--gradient_checkpointing', '--learning_rate=5e-6', '--lr_scheduler=constant', '--lr_warmup_steps=0', '--num_class_images=100', '--sample_batch_size=4', '--max_train_steps=800']' returned non-zero exit status 1.

I don't know if huggingface is having issues right now

Klaudioz avatar Nov 02 '22 20:11 Klaudioz

Could you try upgrading the huggingface_hub library?:

pip install --upgrade huggingface_hub

patrickvonplaten avatar Nov 04 '22 16:11 patrickvonplaten

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

github-actions[bot] avatar Nov 29 '22 15:11 github-actions[bot]

Hey guys! I found a solution(at least for me) to this problem, the issue is caused by the "$" symbol when giving MODEL_NAME (and other env variables). It's working for me if you directly pass the string paths as argument to the 'accelerate launch' command for example:

!accelerate launch train_custom_diffusion.py \ --pretrained_model_name_or_path="CompVis/stable-diffusion-v1-4" \ --instance_data_dir="./data/cat" \ --output_dir="/content" \ --class_data_dir=./real_reg/samples_cat/ \ --with_prior_preservation --prior_loss_weight=1.0 \ --class_prompt="cat" --num_class_images=200 \ --instance_prompt="photo of a <new1> cat" \ --resolution=512 \ --train_batch_size=2 \ --learning_rate=1e-5 \ --lr_warmup_steps=0 \ --max_train_steps=250 \ --scale_lr --hflip \ --modifier_token "<new1>" --class_data_dir=./real_reg/samples_cat/ --with_prior_preservation --prior_loss_weight=1.0 --class_prompt="cat" --num_class_images=200 --instance_prompt="photo of a cat" --resolution=512 --train_batch_size=2 --learning_rate=1e-5 --lr_warmup_steps=0 --max_train_steps=250 --scale_lr --hflip \ --modifier_token "<new1>"

Ali-Hasan-Khan avatar May 26 '23 11:05 Ali-Hasan-Khan

Hey guys! I found a solution(at least for me) to this problem, the issue is caused by the "$" symbol when giving MODEL_NAME (and other env variables). It's working for me if you directly pass the string paths as argument to the 'accelerate launch' command for example:

!accelerate launch train_custom_diffusion.py \ --pretrained_model_name_or_path="CompVis/stable-diffusion-v1-4" \ --instance_data_dir="./data/cat" \ --output_dir="/content" \ --class_data_dir=./real_reg/samples_cat/ \ --with_prior_preservation --prior_loss_weight=1.0 \ --class_prompt="cat" --num_class_images=200 \ --instance_prompt="photo of a <new1> cat" \ --resolution=512 \ --train_batch_size=2 \ --learning_rate=1e-5 \ --lr_warmup_steps=0 \ --max_train_steps=250 \ --scale_lr --hflip \ --modifier_token "<new1>" --class_data_dir=./real_reg/samples_cat/ --with_prior_preservation --prior_loss_weight=1.0 --class_prompt="cat" --num_class_images=200 --instance_prompt="photo of a cat" --resolution=512 --train_batch_size=2 --learning_rate=1e-5 --lr_warmup_steps=0 --max_train_steps=250 --scale_lr --hflip \ --modifier_token "<new1>"

You saved my life. Thanks man :)

kishantrivedi avatar May 28 '23 01:05 kishantrivedi

changing gradio theme: HFValidationError Traceback (most recent call last): File "/Users/xx/sdwebui/modules/shared.py", line 774, in reload_gradio_theme gradio_theme = gr.themes.ThemeClass.from_hub(theme_name) File "/opt/homebrew/lib/python3.10/site-packages/gradio/themes/base.py", line 165, in from_hub space_info = api.space_info(name) File "/opt/homebrew/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 110, in _inner_fn validate_repo_id(arg_value) File "/opt/homebrew/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 164, in validate_repo_id raise HFValidationError( huggingface_hub.utils.validators.HFValidationError: Repo id must use alphanumeric chars or '-', '', '.', '--' and '..' are forbidden, '-' and '.' cannot start or end the name, max length is 96: ''.

madsteed avatar Jul 08 '23 11:07 madsteed

我刚刚遇到了这个问题.几个小时前,我在运行此命令时没有任何问题:

sudo docker run -it --gpus=all --ipc=host -v $(pwd):/root -e HUGGING_FACE_HUB_TOKEN=$TOKEN smy20011/dreambooth:latest accelerate launch /train_dreambooth.py   --pretrained_model_name_or_path=$MODEL_NAME   --instance_data_dir=$INSTANCE_DIR  --class_data_dir=$CLASS_DIR   --output_dir=$OUTPUT_DIR   --with_prior_preservation --prior_loss_weight=1.0   --instance_prompt="a portrait of sks woman"   --class_prompt="a portrait of woman"   --resolution=512   --center_crop   --train_batch_size=1   --mixed_precision="fp16"   --use_8bit_adam   --gradient_accumulation_steps=1 --gradient_checkpointing   --learning_rate=5e-6   --lr_scheduler="constant"   --lr_warmup_steps=0   --num_class_images=100   --sample_batch_size=4   --max_train_steps=800
The following values were not passed to `accelerate launch` and had defaults used instead:
        `--num_processes` was set to a value of `1`
        `--num_machines` was set to a value of `1`
        `--mixed_precision` was set to a value of `'no'`
        `--num_cpu_threads_per_process` was set to `3` to improve out-of-box performance
To avoid this warning pass in values for each of the problematic parameters or run `accelerate config`.
Traceback (most recent call last):
  File "/opt/conda/lib/python3.7/site-packages/diffusers/configuration_utils.py", line 234, in get_config_dict
    revision=revision,
  File "/opt/conda/lib/python3.7/site-packages/huggingface_hub/file_download.py", line 1022, in hf_hub_download
    cache_dir, repo_folder_name(repo_id=repo_id, repo_type=repo_type)
  File "/opt/conda/lib/python3.7/site-packages/huggingface_hub/utils/_validators.py", line 92, in _inner_fn
    validate_repo_id(arg_value)
  File "/opt/conda/lib/python3.7/site-packages/huggingface_hub/utils/_validators.py", line 143, in validate_repo_id
    "Repo id must use alphanumeric chars or '-', '_', '.', '--' and '..' are"
huggingface_hub.utils._validators.HFValidationError: Repo id must use alphanumeric chars or '-', '_', '.', '--' and '..' are forbidden, '-' and '.' cannot start or end the name, max length is 96: ''.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/train_dreambooth.py", line 695, in <module>
    main()
  File "/train_dreambooth.py", line 376, in main
    args.pretrained_model_name_or_path, torch_dtype=torch_dtype, use_auth_token=True
  File "/opt/conda/lib/python3.7/site-packages/diffusers/pipeline_utils.py", line 372, in from_pretrained
    revision=revision,
  File "/opt/conda/lib/python3.7/site-packages/diffusers/configuration_utils.py", line 261, in get_config_dict
    f"We couldn't connect to '{HUGGINGFACE_CO_RESOLVE_ENDPOINT}' to load this model, couldn't find it"
OSError: We couldn't connect to 'https://huggingface.co' to load this model, couldn't find it in the cached files and it looks like  is not the path to a directory containing a model_index.json file.
Checkout your internet connection or see how to run the library in offline mode at 'https://huggingface.co/docs/diffusers/installation#offline-mode'.
Traceback (most recent call last):
  File "/opt/conda/bin/accelerate", line 8, in <module>
    sys.exit(main())
  File "/opt/conda/lib/python3.7/site-packages/accelerate/commands/accelerate_cli.py", line 43, in main
    args.func(args)
  File "/opt/conda/lib/python3.7/site-packages/accelerate/commands/launch.py", line 837, in launch_command
    simple_launcher(args)
  File "/opt/conda/lib/python3.7/site-packages/accelerate/commands/launch.py", line 354, in simple_launcher
    raise subprocess.CalledProcessError(returncode=process.returncode, cmd=cmd)
subprocess.CalledProcessError: Command '['/opt/conda/bin/python', '/train_dreambooth.py', '--pretrained_model_name_or_path=', '--instance_data_dir=', '--class_data_dir=', '--output_dir=', '--with_prior_preservation', '--prior_loss_weight=1.0', '--instance_prompt=a portrait of sks woman', '--class_prompt=a portrait of woman', '--resolution=512', '--center_crop', '--train_batch_size=1', '--mixed_precision=fp16', '--use_8bit_adam', '--gradient_accumulation_steps=1', '--gradient_checkpointing', '--learning_rate=5e-6', '--lr_scheduler=constant', '--lr_warmup_steps=0', '--num_class_images=100', '--sample_batch_size=4', '--max_train_steps=800']' returned non-zero exit status 1.

我不知道huggingface现在是否有问题 did you slove?

awaylong avatar Dec 01 '23 18:12 awaylong

the saga continues.. :)

for me leaving out the quotes during the assignment made it work.. something about my vscode jupyter setup was adding extra quotes.. so this way i could keep the variables without needing direct assignment.

`%env MODEL_NAME=stabilityai/stable-diffusion-xl-base-1.0 %env DATASET_NAME=./3d_icon %env OUTPUT_DIR=./lora %env VAE_PATH=madebyollin/sdxl-vae-fp16-fix

!accelerate launch ../examples/advanced_diffusion_training/train_dreambooth_lora_sdxl_advanced.py
--pretrained_model_name_or_path=$MODEL_NAME
--pretrained_vae_model_name_or_path=$VAE_PATH
--dataset_name=$DATASET_NAME
--instance_prompt="3d icon in the style of TOK"
--validation_prompt="a TOK icon of an astronaut riding a horse, in the style of TOK"
--output_dir=$OUTPUT_DIR `

DimitriGeier avatar Mar 18 '24 18:03 DimitriGeier

Hi, Is it possible to get any help here with an error I posted at https://github.com/bmaltais/kohya_ss/issues/2185#issuecomment-2045184732

thanks a lot in advance, this seems to be still kind of an issue on MAC

dajanaelez avatar Apr 20 '24 12:04 dajanaelez

After reading about this issue this is not a problem with diffusers but an environment issue with the users?

Also we cannot help you with that repository because they use an old diffusers version probably to prevent something going wrong on their side:

diffusers[torch]==0.25.0

Even if it was a diffusers problem and we fix it, you wouldn't be able to use it.

Your only option right now is to wait for them to reply to you or just use diffusers without that repository: https://huggingface.co/docs/diffusers/training/dreambooth#lora

asomoza avatar Apr 20 '24 15:04 asomoza