It starts training, but after a few minutes I get this error.
Progress:| | 2% 69/3000 [02:50<1:57:06, 2.40s/it, loss=0.477, lr=1.96e-6]Traceback (most recent call last):
File "/content/diffusers/examples/dreambooth/train_dreambooth.py", line 798, in
main()
File "/content/diffusers/examples/dreambooth/train_dreambooth.py", line 632, in main
for step, batch in enumerate(train_dataloader):
File "/usr/local/lib/python3.8/dist-packages/accelerate/data_loader.py", line 357, in iter
next_batch = next(dataloader_iter)
File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/dataloader.py", line 681, in next
data = self._next_data()
File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/dataloader.py", line 721, in _next_data
data = self._dataset_fetcher.fetch(index) # may raise StopIteration
File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/_utils/fetch.py", line 49, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/_utils/fetch.py", line 49, in
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/content/diffusers/examples/dreambooth/train_dreambooth.py", line 324, in getitem
instance_image = Image.open(path)
File "/usr/local/lib/python3.8/dist-packages/PIL/Image.py", line 2843, in open
fp = builtins.open(filename, "rb")
IsADirectoryError: [Errno 21] Is a directory: '/content/gdrive/MyDrive/Fast-Dreambooth/Sessions/ndl/instance_images/.ipynb_checkpoints'
Progress:| | 2% 69/3000 [02:50<2:00:58, 2.48s/it, loss=0.477, lr=1.96e-6]
Traceback (most recent call last):
File "/usr/local/bin/accelerate", line 8, in
sys.exit(main())
File "/usr/local/lib/python3.8/dist-packages/accelerate/commands/accelerate_cli.py", line 43, in main
args.func(args)
File "/usr/local/lib/python3.8/dist-packages/accelerate/commands/launch.py", line 837, in launch_command
simple_launcher(args)
File "/usr/local/lib/python3.8/dist-packages/accelerate/commands/launch.py", line 354, in simple_launcher
raise subprocess.CalledProcessError(returncode=process.returncode, cmd=cmd)
subprocess.CalledProcessError: Command '['/usr/bin/python3', '/content/diffusers/examples/dreambooth/train_dreambooth.py', '--train_text_encoder', '--image_captions_filename', '--save_starting_step=500', '--stop_text_encoder_training=900', '--save_n_steps=0', '--Session_dir=/content/gdrive/MyDrive/Fast-Dreambooth/Sessions/ndl', '--pretrained_model_name_or_path=/content/stable-diffusion-v1-5', '--instance_data_dir=/content/gdrive/MyDrive/Fast-Dreambooth/Sessions/ndl/instance_images', '--output_dir=/content/models/ndl', '--instance_prompt=', '--seed=455387', '--resolution=768', '--mixed_precision=fp16', '--train_batch_size=1', '--gradient_accumulation_steps=1', '--gradient_checkpointing', '--use_8bit_adam', '--learning_rate=2e-6', '--lr_scheduler=polynomial', '--lr_warmup_steps=0', '--max_train_steps=3000']' returned non-zero exit status 1.
Something went wrong
remove the folder ".ipynb_checkpoints" from /content/gdrive/MyDrive/Fast-Dreambooth/Sessions/ndl/instance_images/
Removing the folder with the path mentioned in the log solved it for me as well.