Stable Diffusion fine-tuning problem

[838959823]

During fine-tuning for depth estimateaion conditioned on input image, how to deal with the text prompt required in the original pre_trained text-to-image Stable Diffusion model?

[saqib736]

As far as i understand, you can just pass the empty prompts.

[billie2jean]

I cannot understand the fine-tuning, i think it is a process of pre-training for getting the CKPT, and also not find the part corresponding to fine-tuning in the code. I wanna ask for your help, pls.

[saqib736]

I cannot understand the fine-tuning, i think it is a process of pre-training for getting the CKPT, and also not find the part corresponding to fine-tuning in the code. I wanna ask for your help, pls.

Fine-tuning means training the pre-trained stable diffusion model on your specific dataset. This repository doesn't contain the training code yet; only the testing/inference code is available.

If you want to train or fine-tune the model on your dataset or any other dataset, I have written a simple training code for Marigold. You can find it on my GitHub here: training code for Marigold.

[billie2jean]

Thx, :)

[markkua]

Training code has been released.