您好我这边在数据处理阶段用的指令如下:
CUDA_VISIBLE_DEVICES="0" python ${project_dir}/examples/wanvideo/train_wan_t2v.py
--task data_process
--dataset_path $data_dir
--output_path ./models
--text_encoder_path "$bash_checkpoint/models_t5_umt5-xxl-enc-bf16.pth"
--vae_path "$bash_checkpoint/Wan2.1_VAE.pth"
--image_encoder_path "$bash_checkpoint/models_clip_open-clip-xlm-roberta-large-vit-huge-14.pth"
--num_frames 144
--height 720
--width 1280
--tiled
报错:
File "/mnt/workspace/guiji.hc/code/DiffSynth-Studio-main/examples/wanvideo/train_wan_t2v.py", line 599, in
data_process(args)
File "/mnt/workspace/guiji.hc/code/DiffSynth-Studio-main/examples/wanvideo/train_wan_t2v.py", line 541, in data_process
trainer.test(model, dataloader)
File "/mnt/workspace/guiji.hc/pythonEnv/diffstudio/lib/python3.10/site-packages/lightning/pytorch/trainer/trainer.py", line 775, in test
return call._call_and_handle_interrupt(
File "/mnt/workspace/guiji.hc/pythonEnv/diffstudio/lib/python3.10/site-packages/lightning/pytorch/trainer/call.py", line 48, in _call_and_handle_interrupt
return trainer_fn(*args, **kwargs)
File "/mnt/workspace/guiji.hc/pythonEnv/diffstudio/lib/python3.10/site-packages/lightning/pytorch/trainer/trainer.py", line 817, in _test_impl
results = self._run(model, ckpt_path=ckpt_path)
File "/mnt/workspace/guiji.hc/pythonEnv/diffstudio/lib/python3.10/site-packages/lightning/pytorch/trainer/trainer.py", line 1012, in _run
results = self._run_stage()
File "/mnt/workspace/guiji.hc/pythonEnv/diffstudio/lib/python3.10/site-packages/lightning/pytorch/trainer/trainer.py", line 1049, in _run_stage
return self._evaluation_loop.run()
File "/mnt/workspace/guiji.hc/pythonEnv/diffstudio/lib/python3.10/site-packages/lightning/pytorch/loops/utilities.py", line 179, in _decorator
return loop_run(self, *args, **kwargs)
File "/mnt/workspace/guiji.hc/pythonEnv/diffstudio/lib/python3.10/site-packages/lightning/pytorch/loops/evaluation_loop.py", line 145, in run
self._evaluation_step(batch, batch_idx, dataloader_idx, dataloader_iter)
File "/mnt/workspace/guiji.hc/pythonEnv/diffstudio/lib/python3.10/site-packages/lightning/pytorch/loops/evaluation_loop.py", line 437, in _evaluation_step
output = call._call_strategy_hook(trainer, hook_name, *step_args)
File "/mnt/workspace/guiji.hc/pythonEnv/diffstudio/lib/python3.10/site-packages/lightning/pytorch/trainer/call.py", line 328, in _call_strategy_hook
output = fn(*args, **kwargs)
File "/mnt/workspace/guiji.hc/pythonEnv/diffstudio/lib/python3.10/site-packages/lightning/pytorch/strategies/strategy.py", line 425, in test_step
return self.lightning_module.test_step(*args, **kwargs)
File "/mnt/workspace/guiji.hc/code/DiffSynth-Studio-main/examples/wanvideo/train_wan_t2v.py", line 152, in test_step
image_emb = self.pipe.encode_image(first_frame, None, num_frames, height, width)
File "/mnt/workspace/guiji.hc/code/DiffSynth-Studio-main/diffsynth/pipelines/wan_video.py", line 221, in encode_image
msk = msk.view(1, msk.shape[1] // 4, 4, height//8, width//8)
RuntimeError: shape '[1, 36, 4, 90, 160]' is invalid for input of size 2116800
请问是否支持720P&30帧率的图生视频任务,现在好像只能训练480p&16帧率,感谢~