图像合成的推理和微调报错

[luoan7248]

在设置微调和推理的图像尺寸参数时，是否有什么特殊的约定。

在训练 lora 时，高度设置为 1024、宽度设置为 560 会报错，推理也是这样设置的，也会报错。

但如果同时设置 1024，就不会出错。

具体信息如下

Epoch 0: 0%| | 0/250 [00:00<?, ?it/s]Traceback (most recent call last): File "examples/train/kolors/train_kolors_lora.py", line 77, in launch_training_task(model, args) File "/workspace/luoan/DiffSynth-Studio-main/diffsynth/trainers/text_to_image.py", line 244, in launch_training_task trainer.fit(model=model, train_dataloaders=train_loader) File "/mnt/10101/staryea/anaconda3/envs/scepter/lib/python3.8/site-packages/lightning/pytorch/trainer/trainer.py", line 543, in fit call._call_and_handle_interrupt( File "/mnt/10101/staryea/anaconda3/envs/scepter/lib/python3.8/site-packages/lightning/pytorch/trainer/call.py", line 43, in _call_and_handle_interrupt return trainer.strategy.launcher.launch(trainer_fn, *args, trainer=trainer, **kwargs) File "/mnt/10101/staryea/anaconda3/envs/scepter/lib/python3.8/site-packages/lightning/pytorch/strategies/launchers/subprocess_script.py", line 105, in launch return function(*args, **kwargs) File "/mnt/10101/staryea/anaconda3/envs/scepter/lib/python3.8/site-packages/lightning/pytorch/trainer/trainer.py", line 579, in _fit_impl self._run(model, ckpt_path=ckpt_path) File "/mnt/10101/staryea/anaconda3/envs/scepter/lib/python3.8/site-packages/lightning/pytorch/trainer/trainer.py", line 986, in _run results = self._run_stage() File "/mnt/10101/staryea/anaconda3/envs/scepter/lib/python3.8/site-packages/lightning/pytorch/trainer/trainer.py", line 1030, in _run_stage self.fit_loop.run() File "/mnt/10101/staryea/anaconda3/envs/scepter/lib/python3.8/site-packages/lightning/pytorch/loops/fit_loop.py", line 205, in run self.advance() File "/mnt/10101/staryea/anaconda3/envs/scepter/lib/python3.8/site-packages/lightning/pytorch/loops/fit_loop.py", line 363, in advance self.epoch_loop.run(self._data_fetcher) File "/mnt/10101/staryea/anaconda3/envs/scepter/lib/python3.8/site-packages/lightning/pytorch/loops/training_epoch_loop.py", line 140, in run self.advance(data_fetcher) File "/mnt/10101/staryea/anaconda3/envs/scepter/lib/python3.8/site-packages/lightning/pytorch/loops/training_epoch_loop.py", line 250, in advance batch_output = self.automatic_optimization.run(trainer.optimizers[0], batch_idx, kwargs) File "/mnt/10101/staryea/anaconda3/envs/scepter/lib/python3.8/site-packages/lightning/pytorch/loops/optimization/automatic.py", line 190, in run self._optimizer_step(batch_idx, closure) File "/mnt/10101/staryea/anaconda3/envs/scepter/lib/python3.8/site-packages/lightning/pytorch/loops/optimization/automatic.py", line 268, in _optimizer_step call._call_lightning_module_hook( File "/mnt/10101/staryea/anaconda3/envs/scepter/lib/python3.8/site-packages/lightning/pytorch/trainer/call.py", line 159, in _call_lightning_module_hook output = fn(*args, **kwargs) File "/mnt/10101/staryea/anaconda3/envs/scepter/lib/python3.8/site-packages/lightning/pytorch/core/module.py", line 1308, in optimizer_step optimizer.step(closure=optimizer_closure) File "/mnt/10101/staryea/anaconda3/envs/scepter/lib/python3.8/site-packages/lightning/pytorch/core/optimizer.py", line 153, in step step_output = self._strategy.optimizer_step(self._optimizer, closure, **kwargs) File "/mnt/10101/staryea/anaconda3/envs/scepter/lib/python3.8/site-packages/lightning/pytorch/strategies/ddp.py", line 270, in optimizer_step optimizer_output = super().optimizer_step(optimizer, closure, model, **kwargs) File "/mnt/10101/staryea/anaconda3/envs/scepter/lib/python3.8/site-packages/lightning/pytorch/strategies/strategy.py", line 238, in optimizer_step return self.precision_plugin.optimizer_step(optimizer, model=model, closure=closure, **kwargs) File "/mnt/10101/staryea/anaconda3/envs/scepter/lib/python3.8/site-packages/lightning/pytorch/plugins/precision/amp.py", line 77, in optimizer_step closure_result = closure() File "/mnt/10101/staryea/anaconda3/envs/scepter/lib/python3.8/site-packages/lightning/pytorch/loops/optimization/automatic.py", line 144, in call self._result = self.closure(*args, **kwargs) File "/mnt/10101/staryea/anaconda3/envs/scepter/lib/python3.8/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(*args, **kwargs) File "/mnt/10101/staryea/anaconda3/envs/scepter/lib/python3.8/site-packages/lightning/pytorch/loops/optimization/automatic.py", line 129, in closure step_output = self._step_fn() File "/mnt/10101/staryea/anaconda3/envs/scepter/lib/python3.8/site-packages/lightning/pytorch/loops/optimization/automatic.py", line 317, in _training_step training_step_output = call._call_strategy_hook(trainer, "training_step", *kwargs.values()) File "/mnt/10101/staryea/anaconda3/envs/scepter/lib/python3.8/site-packages/lightning/pytorch/trainer/call.py", line 311, in _call_strategy_hook output = fn(*args, **kwargs) File "/mnt/10101/staryea/anaconda3/envs/scepter/lib/python3.8/site-packages/lightning/pytorch/strategies/strategy.py", line 389, in training_step return self._forward_redirection(self.model, self.lightning_module, "training_step", *args, **kwargs) File "/mnt/10101/staryea/anaconda3/envs/scepter/lib/python3.8/site-packages/lightning/pytorch/strategies/strategy.py", line 640, in call wrapper_output = wrapper_module(*args, **kwargs) File "/mnt/10101/staryea/anaconda3/envs/scepter/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, **kwargs) File "/mnt/10101/staryea/anaconda3/envs/scepter/lib/python3.8/site-packages/torch/nn/parallel/distributed.py", line 1156, in forward output = self._run_ddp_forward(*inputs, **kwargs) File "/mnt/10101/staryea/anaconda3/envs/scepter/lib/python3.8/site-packages/torch/nn/parallel/distributed.py", line 1110, in _run_ddp_forward return module_to_run(*inputs[0], **kwargs[0]) # type: ignore[index] File "/mnt/10101/staryea/anaconda3/envs/scepter/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(args, **kwargs) File "/mnt/10101/staryea/anaconda3/envs/scepter/lib/python3.8/site-packages/lightning/pytorch/strategies/strategy.py", line 633, in wrapped_forward out = method(_args, **_kwargs) File "/workspace/luoan/DiffSynth-Studio-main/diffsynth/trainers/text_to_image.py", line 64, in training_step noise_pred = self.pipe.denoising_model()( File "/mnt/10101/staryea/anaconda3/envs/scepter/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, **kwargs) File "/workspace/luoan/DiffSynth-Studio-main/diffsynth/models/sdxl_unet.py", line 126, in forward hidden_states, time_emb, text_emb, res_stack = block( File "/mnt/10101/staryea/anaconda3/envs/scepter/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, **kwargs) File "/workspace/luoan/DiffSynth-Studio-main/diffsynth/models/sd_unet.py", line 226, in forward hidden_states = torch.cat([hidden_states, res_hidden_states], dim=1) RuntimeError: Sizes of tensors must match except in dimension 1. Expected size 36 but got size 35 for tensor number 1 in the list. Traceback (most recent call last): File "/workspace/luoan/DiffSynth-Studio-main/examples/train/kolors/train_kolors_lora.py", line 77, in launch_training_task(model, args) File "/workspace/luoan/DiffSynth-Studio-main/diffsynth/trainers/text_to_image.py", line 244, in launch_training_task trainer.fit(model=model, train_dataloaders=train_loader) File "/mnt/10101/staryea/anaconda3/envs/scepter/lib/python3.8/site-packages/lightning/pytorch/trainer/trainer.py", line 543, in fit call._call_and_handle_interrupt( File "/mnt/10101/staryea/anaconda3/envs/scepter/lib/python3.8/site-packages/lightning/pytorch/trainer/call.py", line 43, in _call_and_handle_interrupt return trainer.strategy.launcher.launch(trainer_fn, *args, trainer=trainer, **kwargs) File "/mnt/10101/staryea/anaconda3/envs/scepter/lib/python3.8/site-packages/lightning/pytorch/strategies/launchers/subprocess_script.py", line 105, in launch return function(*args, **kwargs) File "/mnt/10101/staryea/anaconda3/envs/scepter/lib/python3.8/site-packages/lightning/pytorch/trainer/trainer.py", line 579, in _fit_impl self._run(model, ckpt_path=ckpt_path) File "/mnt/10101/staryea/anaconda3/envs/scepter/lib/python3.8/site-packages/lightning/pytorch/trainer/trainer.py", line 986, in _run results = self._run_stage() File "/mnt/10101/staryea/anaconda3/envs/scepter/lib/python3.8/site-packages/lightning/pytorch/trainer/trainer.py", line 1030, in _run_stage self.fit_loop.run() File "/mnt/10101/staryea/anaconda3/envs/scepter/lib/python3.8/site-packages/lightning/pytorch/loops/fit_loop.py", line 205, in run self.advance() File "/mnt/10101/staryea/anaconda3/envs/scepter/lib/python3.8/site-packages/lightning/pytorch/loops/fit_loop.py", line 363, in advance self.epoch_loop.run(self._data_fetcher) File "/mnt/10101/staryea/anaconda3/envs/scepter/lib/python3.8/site-packages/lightning/pytorch/loops/training_epoch_loop.py", line 140, in run self.advance(data_fetcher) File "/mnt/10101/staryea/anaconda3/envs/scepter/lib/python3.8/site-packages/lightning/pytorch/loops/training_epoch_loop.py", line 250, in advance batch_output = self.automatic_optimization.run(trainer.optimizers[0], batch_idx, kwargs) File "/mnt/10101/staryea/anaconda3/envs/scepter/lib/python3.8/site-packages/lightning/pytorch/loops/optimization/automatic.py", line 190, in run self._optimizer_step(batch_idx, closure) File "/mnt/10101/staryea/anaconda3/envs/scepter/lib/python3.8/site-packages/lightning/pytorch/loops/optimization/automatic.py", line 268, in _optimizer_step call._call_lightning_module_hook( File "/mnt/10101/staryea/anaconda3/envs/scepter/lib/python3.8/site-packages/lightning/pytorch/trainer/call.py", line 159, in _call_lightning_module_hook output = fn(*args, **kwargs) File "/mnt/10101/staryea/anaconda3/envs/scepter/lib/python3.8/site-packages/lightning/pytorch/core/module.py", line 1308, in optimizer_step optimizer.step(closure=optimizer_closure) File "/mnt/10101/staryea/anaconda3/envs/scepter/lib/python3.8/site-packages/lightning/pytorch/core/optimizer.py", line 153, in step step_output = self._strategy.optimizer_step(self._optimizer, closure, **kwargs) File "/mnt/10101/staryea/anaconda3/envs/scepter/lib/python3.8/site-packages/lightning/pytorch/strategies/ddp.py", line 270, in optimizer_step optimizer_output = super().optimizer_step(optimizer, closure, model, **kwargs) File "/mnt/10101/staryea/anaconda3/envs/scepter/lib/python3.8/site-packages/lightning/pytorch/strategies/strategy.py", line 238, in optimizer_step return self.precision_plugin.optimizer_step(optimizer, model=model, closure=closure, **kwargs) File "/mnt/10101/staryea/anaconda3/envs/scepter/lib/python3.8/site-packages/lightning/pytorch/plugins/precision/amp.py", line 77, in optimizer_step closure_result = closure() File "/mnt/10101/staryea/anaconda3/envs/scepter/lib/python3.8/site-packages/lightning/pytorch/loops/optimization/automatic.py", line 144, in call self._result = self.closure(*args, **kwargs) File "/mnt/10101/staryea/anaconda3/envs/scepter/lib/python3.8/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(*args, **kwargs) File "/mnt/10101/staryea/anaconda3/envs/scepter/lib/python3.8/site-packages/lightning/pytorch/loops/optimization/automatic.py", line 129, in closure step_output = self._step_fn() File "/mnt/10101/staryea/anaconda3/envs/scepter/lib/python3.8/site-packages/lightning/pytorch/loops/optimization/automatic.py", line 317, in _training_step training_step_output = call._call_strategy_hook(trainer, "training_step", *kwargs.values()) File "/mnt/10101/staryea/anaconda3/envs/scepter/lib/python3.8/site-packages/lightning/pytorch/trainer/call.py", line 311, in _call_strategy_hook output = fn(*args, **kwargs) File "/mnt/10101/staryea/anaconda3/envs/scepter/lib/python3.8/site-packages/lightning/pytorch/strategies/strategy.py", line 389, in training_step return self._forward_redirection(self.model, self.lightning_module, "training_step", *args, **kwargs) File "/mnt/10101/staryea/anaconda3/envs/scepter/lib/python3.8/site-packages/lightning/pytorch/strategies/strategy.py", line 640, in call wrapper_output = wrapper_module(*args, **kwargs) File "/mnt/10101/staryea/anaconda3/envs/scepter/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, **kwargs) File "/mnt/10101/staryea/anaconda3/envs/scepter/lib/python3.8/site-packages/torch/nn/parallel/distributed.py", line 1156, in forward output = self._run_ddp_forward(*inputs, **kwargs) File "/mnt/10101/staryea/anaconda3/envs/scepter/lib/python3.8/site-packages/torch/nn/parallel/distributed.py", line 1110, in _run_ddp_forward return module_to_run(*inputs[0], **kwargs[0]) # type: ignore[index] File "/mnt/10101/staryea/anaconda3/envs/scepter/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(args, **kwargs) File "/mnt/10101/staryea/anaconda3/envs/scepter/lib/python3.8/site-packages/lightning/pytorch/strategies/strategy.py", line 633, in wrapped_forward out = method(_args, **_kwargs) File "/workspace/luoan/DiffSynth-Studio-main/diffsynth/trainers/text_to_image.py", line 64, in training_step noise_pred = self.pipe.denoising_model()( File "/mnt/10101/staryea/anaconda3/envs/scepter/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, **kwargs) File "/workspace/luoan/DiffSynth-Studio-main/diffsynth/models/sdxl_unet.py", line 126, in forward hidden_states, time_emb, text_emb, res_stack = block( File "/mnt/10101/staryea/anaconda3/envs/scepter/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, **kwargs) File "/workspace/luoan/DiffSynth-Studio-main/diffsynth/models/sd_unet.py", line 226, in forward hidden_states = torch.cat([hidden_states, res_hidden_states], dim=1) RuntimeError: Sizes of tensors must match except in dimension 1. Expected size 36 but got size 35 for tensor number 1 in the list.