Request - FaceRestoration Only on Swapped Face

[Kinglord]

Hello,

I'm using the newest version of the plugin and the improvements are great but are causing me more problems than they have solved. I am trying to replace faces on one person in a crowd, and whenever I use the face restoration feature is applies to all faces it finds in the image, even background faces that are meant to be blurry and out of focus. Obviously I can work around this with some editing, but that's not always possible and takes some time.

I'd like to be able to control what the face restoration applies to, ideally just the replaced face, but if I had some form of control, like being able to set a confidence %, a max number of detected faces, I mean there's a lot of ways to set this up and I'll leave it to you to decide if and when you implement something like this. I think at a high level though, most of us don't want to restore every face that might be present on a multi face image.

That said things are working well on single character images with the exception of eye color but I think that's more a fault with the models than anything else.

[Gourieff]

Hello there!

Yes, you're right, that's not a very pleasant thing of the face restoration logic, Codeformer or GFPGAN gets a whole image and restores the every face that can be detected It's implemented like in A1111 In future releases I will try to solve it somehow (via masks or crop+blend), or maybe someone will make a PR Need to look up if there are some nodes that already can provide smth like an inpainting logic with a mask We can generate an image -> set a mask of the area with faces we need to swap -> swap them and restore via ReActor -> and then blend them back in the image 🤔

[Kinglord]

Thanks for the reply! And I'd love to help more if I could but for now I can just throw you some links in case they help. The most popular and extensive is the Impact Pack that has a bunch of masking and inpainting stuff (i use the face detailer all the time). There's an older one i used to use that might be easier to pick apart that does basic face masking and inpainting here - masquerade pack. What I've been playing with now is not using the built in restore and then using something like the face detailer from impact pack to restore the face I want, it's not bad but it does lose something, but I'm still trying to mess with it but it could be an overall workaround for this and you don't need to worry about it :D

That said I know some other tools that leverage insight face do some detection sorting in terms of location/size - like top/down, left/right, large/small. If that's already built in just being able to tell it to which ones to restore would be easiest but I'm guessing the output from insightface doesn't really work like that, if not, then if we could build a mask based on the output at least that would be pretty baller.

[ckao10301]

Hi @Gourieff I updated the reactor node but am still having this issue. What is the proper workflow to fix this? Mine is like this