Dreambooth-Stable-Diffusion
Dreambooth-Stable-Diffusion copied to clipboard
XavierXiao
"Sks" is a bad identifier. It's a gun.
Using sks, you'll get guns in your generations.
Here's how the original DreamBooth authors found good identifiers:
In a nutshell, our approach is to find relatively rare tokens in the vocabulary, and
then invert these rare tokens into text space. In order to do this, we first perform a rare-token lookup in the
vocabulary and obtain a sequence of rare token identifiers f(Vˆ ), where f is a tokenizer; a function that maps
character sequences to tokens and Vˆ is the decoded text stemming from the tokens f(Vˆ ). This sequence can
be of variable length k with k being a hyperparameter of our method. We find that relatively short sequences
of k = {1, ..., 3} work well. Then, by inverting the vocabulary using the de-tokenizer on f(Vˆ ) we obtain a
sequence of characters that define our unique identifier Vˆ . We observe that using uniform random sampling
without replacement of tokens that correspond to 3 or fewer Unicode characters (without spaces) and using
tokens in the T5-XXL tokenizer range of {5000, ..., 10000} works well.
The DreamBooth authors unfortunately declined to provide what good identifiers they found using their technique. "Sks", I understand, was selected without much scrutiny.
Instead of settling for the problematic "sks", we should instead replicate what the DreamBooth authors did (or maybe just ask them), to find a genuinely good unique identifier.
I lack the domain knowledge to do this myself.
A bit OT, but can we have multiple persons in one model? like sks 1 sks 2 sks 3?
No wonder I kept getting guns in my pics! :-) I use t@y now and it seem to do the trick.
Could never get multiple trainings working well, meaning train on one, then add another training to the same model, etc... But I did get training on multiple people like Couples in a single training. You can't specify the people individually but it knows who is a girl and who is a boy and most images include both, some include one or the other and some include multiples of each. Nothing special in the process except good couple training pics.
what about sks? why you use Sks as identifier？ i think Sks is too similar to sks. Maybe that is language swift?
