kohya-ss
[SDXL] Train SDXL refiner for subject generation
I've been using the scripts here to fine tune the base SDXL model for subject driven generation to good effect. However, I've found that adding the refiner step usually means that the refiner doesn't understand the subject, which often makes using the refiner worse with subject generation.
Currently the scripts do not support fine tuning the refiner to make generation better. When tuning the refiner, realistically we would only need to fine tune the text encoder, not the u-net. If this ability was added, I think that would be very beneficial.
Agree! Training Refiner model is really important now if community want to get a good anime model like NAI. They must train a anime refiner, only base fine-tune is not enough.
+1 to this. SDXL base 1.0 is good, but the refiner is what elevates it to great.
