Load pretrain controlnet error

[rainsoulsrx]

hi, I am training my own model, I want to load pretrain controlnet use "--controlnet_model_name_or_path", but there is an error:

Traceback (most recent call last): File "train_seesr.py", line 1000, in down_block_res_samples, mid_block_res_sample = controlnet( File "/opt/conda/envs/seesr/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, **kwargs) File "/opt/conda/envs/seesr/lib/python3.8/site-packages/accelerate/utils/operations.py", line 687, in forward return model_forward(*args, **kwargs) File "/opt/conda/envs/seesr/lib/python3.8/site-packages/accelerate/utils/operations.py", line 675, in call return convert_to_fp32(self.model_forward(*args, **kwargs)) File "/opt/conda/envs/seesr/lib/python3.8/site-packages/torch/amp/autocast_mode.py", line 14, in decorate_autocast return func(*args, **kwargs) File "/XXX//SeeSR-main/models/controlnet.py", line 766, in forward sample, res_samples = downsample_block( File "/opt/conda/envs/seesr/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, **kwargs) File "/XXX//SeeSR-main/models/unet_2d_blocks.py", line 1238, in forward hidden_states = attn( File "/opt/conda/envs/seesr/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, **kwargs) File "/opt/conda/envs/seesr/lib/python3.8/site-packages/diffusers/models/transformer_2d.py", line 315, in forward hidden_states = block( File "/opt/conda/envs/seesr/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, **kwargs) File "/opt/conda/envs/seesr/lib/python3.8/site-packages/diffusers/models/attention.py", line 218, in forward attn_output = self.attn2( File "/opt/conda/envs/seesr/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, **kwargs) File "/opt/conda/envs/seesr/lib/python3.8/site-packages/diffusers/models/attention_processor.py", line 420, in forward return self.processor( File "/opt/conda/envs/seesr/lib/python3.8/site-packages/diffusers/models/attention_processor.py", line 948, in call key = attn.to_k(encoder_hidden_states, scale=scale) File "/opt/conda/envs/seesr/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, **kwargs) File "/opt/conda/envs/seesr/lib/python3.8/site-packages/diffusers/models/lora.py", line 224, in forward out = super().forward(hidden_states) File "/opt/conda/envs/seesr/lib/python3.8/site-packages/torch/nn/modules/linear.py", line 114, in forward return F.linear(input, self.weight, self.bias) RuntimeError: mat1 and mat2 shapes cannot be multiplied (1232x1024 and 768x320) Steps: 0%| | 0/8000 [00:02<?, ?it/s]