InstantStyle icon indicating copy to clipboard operation
InstantStyle copied to clipboard
verified publisher icon InstantStyle

multi-image support in img2img diffusers

Open DoosooChang opened this issue 1 year ago • 2 comments

Does InstantStyle support referencing multiple images with single ip-adapter for img2img pipeline in diffusers or InstantStyle library?

Multiple image-based txt2img works great in diffusers, but not img2img.

i am using diffusers==0.28.0.dev0

can anyone help me with some working examples?

DoosooChang avatar May 28 '24 04:05 DoosooChang

I believe current IPA is not suitable for img2img pipeline, it's hard to control the strength of init image and ref image (especially when we only inject into a single layer). We are working on improving content preserving ability for img2img style transfer in our next work.

haofanwang avatar May 28 '24 09:05 haofanwang

The img2img pipeline in diffusers doesn't work? (from e-mail)

The img2img pipeline does not produce an error, but when multiple images are inserted into ip_adapter_image (or ip_adapter_imge_embeds by embedding), the resulting image becomes faded and blurry no matter what data it is tested with. Below imgs are exemplary results for single-img and multi-imgs instantstyle, respectively, for oil painting and sunset style transfer.

single:

  • oil painting the man is sitting in the rear of his suv_0_c05_s08

  • sunset a bridge over the water with cars driving_0_c05_s08

multi:

  • oil painting the man is sitting in the rear of his suv_multi_0

  • sunset a bridge over the water with cars driving_multi_0

thank you

DoosooChang avatar Jun 03 '24 02:06 DoosooChang