imaginAIry
imaginAIry copied to clipboard
brycedrennan
Include CrossAttentionControl as an alternative to clipseg
https://github.com/bloc97/CrossAttentionControl/
By the looks of it, this technique seems more intuitive and easier for beginners to use to get the desired output from SD, compared to mask prompts/images with clipseg. I feel like it could fit in well with this project as the goal of both seem to be ease of use.
Is this something that sounds reasonable, or am I overestimating the usefulness of this technique?
The initial and target images must be generated with the same seed for cross attention control to work.
Apparently only works if you're modifying an image you generated, so it's not as useful as something that works on any image. Still, would be great to poke around in the code and see if it can be adapted.
The original paper that repo is based on does indicate there is a way to apply it to normal images. Would be great to find someone who has implemented that part of it. Eventually I may get around to implementing it myself.
I haven't seen any implementations of this and I don't think I'll get around to it. Clipseg appears to work well enough for now.