CogVideo icon indicating copy to clipboard operation
CogVideo copied to clipboard

Finetune stuck at vae encode

Open foreverpiano opened this issue 4 months ago • 2 comments

System Info / 系統信息

11.8 pytorch 2.5.0

Information / 问题信息

  • [ ] The official example scripts / 官方的示例脚本
  • [ ] My own modified scripts / 我自己修改的脚本和任务

Reproduction / 复现过程

def encode_video(video):
    video = video.to(accelerator.device, dtype=vae.dtype).unsqueeze(0)
    video = video.permute(0, 2, 1, 3, 4)  # [B, C, F, H, W]
    latent_dist = vae.encode(video).latent_dist
    return latent_dist
train_dataset.instance_videos = [encode_video(video) for video in train_dataset.instance_videos]

this fuction stuck https://github.com/THUDM/CogVideo/blob/main/finetune/train_cogvideox_lora.py

Expected behavior / 期待表现

workr normal

foreverpiano avatar Oct 12 '24 15:10 foreverpiano