Weird training results

[KIMGEONUNG]

Hi, thank you for your great work. I hava a question for the ControlNet training

I am just trying to train ControlNet with canny edge condition using OpenImage Dataset to reproduce or mimic your traning scheme. I preserve all training setting with the tutorial you gave us. However, the results is so weird like below and that was not recovered even after many training iterations.

Did you have a experience like me during training? or is there any opinion to this phenomenon? I just want to get some opinion or intuition about this from you.

Thank you.

The prompt is fixed by "a high-quality, detailed, and professional image"

Input

Ourput

Recon

[lllyasviel]

generate paired prompts and then use larger batch size (perhaps by accumulation)

[wenyihong]

generate paired prompts and then use larger batch size (perhaps by accumulation)

What's the appropriate batch size, from your experience?

[kings-rgb]

generate paired prompts and then use larger batch size (perhaps by accumulation)

What's the appropriate batch size, from your experience?

Do you solve this problem?

[KIMGEONUNG]

generate paired prompts and then use larger batch size (perhaps by accumulation)

What's the appropriate batch size, from your experience?

Do you solve this problem?

No, after many experiments, I tentatively concluded that the training code has unknown bug.