[BUG]: IndexError: deque index out of range

[winktool]

🐛 Describe the bug

When i used https://github.com/hpcaitech/ColossalAI/tree/main/examples/images/dreambooth , i got this error: IndexError: deque index out of range

1.4 works:

export MODEL_NAME="CompVis/stable-diffusion-v1-4"
export INSTANCE_DIR="/home/hero/datasets/fluff/data"
export OUTPUT_DIR="/home/hero/dream"

torchrun --nproc_per_node 3 train_dreambooth_colossalai.py \
  --pretrained_model_name_or_path=$MODEL_NAME  \
  --instance_data_dir=$INSTANCE_DIR \
  --output_dir=$OUTPUT_DIR \
  --instance_prompt="art of fluff" \
  --resolution=512 \
  --train_batch_size=1 \
  --learning_rate=5e-6 \
  --lr_scheduler="constant" \
  --lr_warmup_steps=0 \
  --max_train_steps=400 \
  --placement="cuda"

2.1 isn't work:

export MODEL_NAME="stabilityai/stable-diffusion-2"
export INSTANCE_DIR="/home/hero/datasets/fluff7681"
export OUTPUT_DIR="/home/hero/dream_fluff"

torchrun --nproc_per_node 3 train_dreambooth_colossalai.py \
  --pretrained_model_name_or_path=$MODEL_NAME  \
  --instance_data_dir=$INSTANCE_DIR \
  --output_dir=$OUTPUT_DIR \
  --instance_prompt="art of fluff" \
  --resolution=768 \
  --train_batch_size=1 \
  --learning_rate=5e-6 \
  --lr_scheduler="constant" \
  --lr_warmup_steps=0 \
  --max_train_steps=400 \
  --placement="cuda"

Environment

latest