TheLastBen
since introducing training multiple subjects, this repo and shiv repo gets bad results
Me and other people noticed that new code introduced in last 2 days now makes the results worse than it was before , something happened maybe in the main diffusers branch, its hard to tell what but the difference is noticeable, subjects likeness is all over the place, fatter faces and not really that close, its pretty weird
Maybe it's related to change with learning rate " - --learning_rate=2e-6 \\n", "+ --learning_rate=1e-6 \\n",
You'd have to swap manually learning rate back and see if thats the culprit
the text encoder training doesn't make thing worse, it makes things more efficient, you don't need 4000 steps anymore, you only need 1600 steps, if you go over it you will overtrain it
I trained it on 2 subjects :
@nfrhtp overtraining will lead to generating exactly the images used for training
well, i keep track of overtraining on shiv repo easily by looking at the 4 samples after model gets saved each 400 steps until 3200 and it takes like 10 seconds to see if the likeness and stylisation are ready, on this repo to do that and run the webui to test models is taking a lot of time. I noticed on his repo with 20 imgs and 2400 steps i get amost overfitting but still very stylisable , so thats the sweet spot for me atm with his rate, but with this repo i used 1700 and it did not work with all subjects so im still looking for best settings. Especialy now that its 1e-6 and not 2e-6.
I just did 1500 and its not enough for 2 people , maybe i shoud double the steps so if i used 1500 i should go for 3000 ? but the results are not really good, deformed faces. Defaults definitely dont work for me, why its suddenly just 600 steps ? This is just not enough
I'm saying it's good because I tried it the first two are the default ones and the last two are trained with 600 steps
These look very good, 30 imgs each and 600 steps ? Theres little room for error if i dont change defauts 1e-6 , my files are named person_man_krb(number).jpg
"person_man_krb(number).jpg" that's the problem, with that name, you're destroying the class person and the class man. you must use only the instance name in the files names : krb(1).jpg .... krb(2).jpg .... etc
the whole method is based on the images filenames which should be only one random unknown keyword, fix that and train for 2000 steps use this negative prompt :
((((ugly)))), (((duplicate))), ((morbid)), ((mutilated)), [out of frame], extra fingers, mutated hands, ((poorly drawn hands)), ((poorly drawn face)), (((mutation))), (((deformed))), ((ugly)), blurry, ((bad anatomy)), (((bad proportions))), ((extra limbs)), cloned face, (((disfigured))), out of frame, ugly, extra limbs, (bad anatomy), gross proportions, (malformed limbs), ((missing arms)), ((missing legs)), (((extra arms))), (((extra legs))), mutated hands, (fused fingers), (too many fingers), (((long neck)))
