Would it be possible to inpaint based on a given picture?

[sethupavan12]

Let's say I have object X (say my cat) which I want in place of the dog on the bench. Then, additionally changing my cat with a text prompt. I know this is not directly stable diffusion but would this be possible in any way? The above was generated using inpaiting model + SD.15

[GeekyGhost]

Yes, so what you'll need to do currently is take the image of your cat, remove the background, take the image of the bench over to openoutpaint then you'll shrink the square and delete a secotion of of the middle of the bench. Next you'll use the stamp tool in openoutpaint and place the cat in the blank square. Then use as normal with prompts and etc. Here let me show you.

[sethupavan12]

Thanks for the reply. Certainly, an interesting and smart way to do it but I am exploring the possibility of giving the image of my cat directly to the model so that the manual work can be removed. If this is not yet possible, your way could be the only way to approach this.

[GeekyGhost]

Thanks for the reply. Certainly, an interesting and smart way to do it but I am exploring the possibility of giving the image of my cat directly to the model so that the manual work can be removed. If this is not yet possible, your way could be the only way to approach this.

Ah, yeah, without manually adding the cat you'd have to do something like doing a fine tuning on a model specifically for your cat so that your cat is what's painted in instead when you use certain keywords. I do that as well for main characters that will be reoccurring, but for one offs I'll just remove the background and inpaint or outpaint within and around them.