motion-cosegmentation icon indicating copy to clipboard operation
motion-cosegmentation copied to clipboard

Apply part swapping on selected part of full size video, like DeepFake work?

Open urmit-space opened this issue 3 years ago • 11 comments

How can apply part swapping in this video on a particular portion.? I get a very unwanted result in the whole video. I know in this video we should detect the face in the full-size video then apply the mask on a particular part. maybe, I know the process but don't know where to start. or is that use different method can you please tell that. can you please take time and help with that!!!?

https://user-images.githubusercontent.com/68328018/104272873-148c6300-54c4-11eb-8e5a-9f66f4081d78.mp4

Iron man replace with this vin image

IMG_0093

urmit-space avatar Jan 12 '21 05:01 urmit-space

This is rather complicated, need to write code for frame detection and back Inpainting.

AliaksandrSiarohin avatar Jan 12 '21 18:01 AliaksandrSiarohin

I guess it will be easier to try some exiting solution for this.

AliaksandrSiarohin avatar Jan 12 '21 18:01 AliaksandrSiarohin

This is rather complicated, need to write code for frame detection and back Inpainting.

yes, it takes more time like 4-5 min to create a face-swapping. but get a good result. can you please help me out as you say the existing solution is easier? I don't know where to change it. if you tried before, let me give some help in the solution.

urmit-space avatar Jan 18 '21 06:01 urmit-space

  1. Slice the video into chunks using https://github.com/AliaksandrSiarohin/first-order-model/blob/master/crop-video.py
  2. Also crop image Win Diesel using paint to make it a square image.
  3. Run a face swap for all chunks from 1.
  4. Write a script that take modified chunks and place them back to the original positions (positions is a part of the ffmpeg command).

AliaksandrSiarohin avatar Jan 19 '21 00:01 AliaksandrSiarohin

OK let me try it. in this, we have to first detect the face in the video. divided video in chunks where a face appeared then crop the face portion. after these things applied by reverse sequence. is that correct? crop-video.py has face crop logic or have to do it myself.

urmit-space avatar Jan 19 '21 04:01 urmit-space

Yes, crop video will produce you ffmpeg commands for cropping. Also you may need to tweek default parameters of crop-video, if you want to include all the chunks.

AliaksandrSiarohin avatar Jan 19 '21 04:01 AliaksandrSiarohin

Ok, let me try once. then get to know what challenges are made from my side. will update on this soon.

urmit-space avatar Jan 19 '21 04:01 urmit-space

I get success in this now all things work. here I attach a video. greatee bro!!

https://user-images.githubusercontent.com/68328018/105016113-44111180-5a68-11eb-91ce-558f94d9404b.mp4

but two things I want to change. how can make all these things automatic just one click and want to improve the result? and other I want to change crop box size like want exectly face crop so may result will improve. The last one how can we achieve this type of face swapping. first-order-model-Face-swap GiF I tried many times but not get exactly this type.

urmit-space avatar Jan 19 '21 09:01 urmit-space

@urmit-space Hi, could you share the code about how to place the croped video back to the original positions (positions is a part of the ffmpeg command). It is appreciated If I can received your reply.

Hpjhpjhs avatar Aug 18 '21 02:08 Hpjhpjhs

@Hpjhpjhs you get x, y, and size from the crop frame. you need to add an overlay video on the current video with respect frame.

!ffmpeg -i /content/Data/4.1_original_conan_obrien.mp4 -vf "movie=/content/Data/predictionscrop.mp4, scale=429: -1 [inner]; [in][inner] overlay =736: 130 [out]" /content/Data/completed.mp4

Maybe this will help.

urmit-space avatar Aug 18 '21 03:08 urmit-space

@urmit-space Thank you so much. That's very helpful.

Hpjhpjhs avatar Aug 18 '21 07:08 Hpjhpjhs