custom dataset

[leoShen917]

Congrats on your great work and thank you for releasing the code. I am curious to train my custom dataset but get stuck in the training process.I want to ask some questions in order to make sure i can train my own dataset successfully. The images in my own dataset do not have corresponding captions. When i start training, which will give good results when I choose No prompt, default prompt or Automatic prompt for the input text prompt. Now I use no prompt input, I have now trained 20,000 global steps, but the output image still has no correlation with my conditional input image. But as you mentioned in your paper, the "sudden converge phenomenon" after 5000 to 10000 steps did not happen in my training. So should I pick default hints or auto hints as text input and retrain, or should I let the current model continue to train? Thanks in advance and really looking forward to your reply! @lllyasviel

[lllyasviel]

generate paired prompts and then use larger batch size (perhaps by accumulation)

[Usernamezhx]

which prompt generator tool do you suggest for custom data? thanks

[leoShen917]

which prompt generator tool do you suggest for custom data? thanks

Mentioned in this paper is the use of BLIP

[leoShen917]

hi,@lllyasviel. Excuse me, I would like to ask if it is possible to use this network to repair broken images, such as those with missing parts or unclear, and I wonder if it is possible to provide such precise control. Thanks again!

[geroldmeisinger]

I think you are looking for the Inpaint Controlnet or basic stable diffusion inpaint. Just mask the broken image parts and let the magic happen.

[leoShen917]

@geroldmeisinger Thanks for your reply, i have solved this issue.

[jiangyuhangcn]

@geroldmeisinger Thanks for your reply, i have solved this issue.

Hi! I have encountered the same problem as you, the output image still has no correlation with my conditional input image. How did you solve it? Is it through gradient accumulation? Thanks !!