SimSwap icon indicating copy to clipboard operation
SimSwap copied to clipboard

can no longer detect far faces and somtimes close faces of any person in the scene

Open ozmix opened this issue 2 years ago • 6 comments

I have been trying this out with different videos and images, sometimes it detects the faces and sometimes it does not. before the improved mask update, it uses to recognize the face on all videos and images. now it sometimes cuts out either being too far away and too close, it's very strange. I'm not sure what it could be, however, I do like this project your doing and I believe it can be the best of the best.

ozmix avatar Aug 04 '21 10:08 ozmix

This should be caused by the mask-dectected model could not detect the mask of that face which is too far or too close. You can try to remove the --use_mask augument in the demo code to see what is different.

NNNNAI avatar Aug 06 '21 07:08 NNNNAI

I guess this happens if the mask-based model does not find the face here: https://github.com/neuralchen/SimSwap/blob/1b8f48279570dda55aa0843987d9a7b6ee87af81/util/reverse2original.py#L107

It makes sense that the recall gets worse with an additional filtering step.

woctezuma avatar Aug 06 '21 07:08 woctezuma

gonna try it now see what happens

ozmix avatar Aug 06 '21 11:08 ozmix

gonna try it now see what happens

so I tried it and it has definitely worked to a degree but im gonna try it some more on other videos and images

ozmix avatar Aug 06 '21 11:08 ozmix

I have played around with the current version (of this date posting it) and there are still issues with close up faces. I also reverted the changes back to the date (SimSwap/util/reverse2original.py) before masking was implemented and there were also issues with it. Does anybody know what the issue could be?

lschaupp avatar Jan 04 '22 10:01 lschaupp

I would suggest creating a new Github issue, and showing some examples of the issues with close-up faces.

As you mentioned, the issue may have been always there, in which case there is not much to do, except improve face detection maybe.

woctezuma avatar Jan 04 '22 11:01 woctezuma