stable-diffusion icon indicating copy to clipboard operation
stable-diffusion copied to clipboard

is there a way to interpolate 2 init images?

Open loboere opened this issue 2 years ago • 5 comments

I mean use 2 init images and Strength see the mix between two images

loboere avatar Sep 04 '22 19:09 loboere

@loboere I'm not sure if you can have 2 init images. It explicitly says img2img in their scripts

limesqueezy avatar Sep 18 '22 12:09 limesqueezy

Midjourney has a new remix feature that merges 2 images. But it'd be sweet if we could do something similar with SD.

WeaverOfTheWeb avatar Nov 10 '22 23:11 WeaverOfTheWeb

Just a pure guess: In a text-to-image model like Stable Diffusion, the noisy latent image is de-noised while conditioned on text embeddings. So what Midjourney did here could be using the image embeddings of the two images to condition the de-noising steps.

bhky avatar Nov 14 '22 16:11 bhky

using the image embeddings of the two images to condition the de-noising steps

Yeah, but stable diffusions releases was trained on conditioning by text only :( Here stable diffusion fine-tuned for "image variations" https://github.com/LambdaLabsML/lambda-diffusers#stable-diffusion-image-variations but need more training steps.

Maybe there is CLIP text embeddings <> image embeddings convertion existing...

rskvazh avatar Nov 18 '22 14:11 rskvazh

Seed travel

saphtea avatar Mar 02 '23 00:03 saphtea