fast-stable-diffusion
fast-stable-diffusion copied to clipboard
TheLastBen
Struggling to get any likeness after training model
Hey everyone!
I trained a model today using pictures of my girlfriend (with consent of course). Used the recommended 52 "Instances" images (mix of closeup portraits, waist shots and full shots) + 200 "Class" Images of women in their 20's all pulled from Unsplash. All the pictures were cropped to 1:1 and highlighted varied poses and lighting composition.
Settings I ran with her Subject_Name = Person, Instance_Name= sks, 2500 steps
Examples of the training set
Instances Pictures:
Class Pictures:
--
I've been really disappointed in the results, especially when comparing them to Astraea (which I believe uses Stable Diffusion and Dream-Booth as well, and was trained on the same set of instances. Am I doing something wrong?
--
Examples of Renders:
Render 1
Astraea
VS
Fast SD
--
Render 2
Astraea
VS
Fast SD
--
Render 3
Fast SD
VS
Astraea
--
In general, the art through SD Repo is interesting, but you'll notice the likeness is completely off. I use the "sks person" trigger in both prompts and they are similar in every-way (including CFG scale and number of render steps). I know the training was successful because if I type only "sks person" in SD as a prompt I get a picture of the model it was trained on.
Anyone have any thoughts on why the results are so poor?
I get better results with more CLASS images, between 300 and 400 images (I'm now testing with 1000+). In your case, try using the subject word "woman". I think the number of steps is fine, I have seen people with good results with 1000 steps or less, although I am trying with more steps, something between 2500-4000.
To speed up the loading/generation of the class images, you can try downloading some regularization images and saving them somewhere in gdrive, I use This
I'd suggest just training with class images that were generated by SD as opposed to real-world ones. (like the person_ddim ones). The class images are supposed to (to a point) preserve what SD would normally generate with your class.
You flipped class pictures with instance pictures, try again with the opposite.
You flipped class pictures with instance pictures, try again with the opposite.
They were miss labeled only on this thread. I ran them correctly on the collab. Maybe I need a lot more than 52 class pictures like someone suggested?
@Antoinevdlb the CLASS pictures must be 200+ and instance pictures 50 is fine, try unchecking the fp16 option and rerun the training. here is an example of good results : https://www.reddit.com/r/StableDiffusion/comments/xxh5y6/dreambooth_consistently_blows_me_away_results/
lol i keep getting this wrong sorry...
I definitely have 52 pictures of my GF's face (Instance) and 200 pictures of young women (Class)
When I ran it last time I ran it with fp16 UNCHECKED
Could I need more class pictures? Should I focus only on sharp portraits and forget full body shots? I also worry the sks person trigger doesn't work great potentially
I'll try again, using the full set of 100 pictures I have for instance and the 1000 I have for class. We'll see how it goes.
I have the ability to run this locally (Titan RTX card). Would there be any benefit to that?
don't use sks person, use "young woman" as a CLASS name, and any other identifier as an instance
The important thing is the code, as long as google colab can run the script, you don't need a different hardware
don't use sks person, use "young woman" as a CLASS name, and any other identifier as an instance
In this case, just to make sure i have this right:
Subject name would be "Young Women" and instance name "Jennie" for example?
Yes, but for the instance, it's better to add something to the name eg : "Jennieabcd" to avoid conflict with the Jennies SD already knows
using sks has been making me laugh, the model keeps trying to stick sks guns everywhere lol.