Simswap 512 on Colab

[Lyntai]

Hello!

The results are awesome, I'm trying to reproduce them on Colab but I'm having issues, the code as it is, seems to output as the original behavior.

If I modify crop_size = opt.crop_size to crop_size = 512 nothing happens.

I I use instead use: opt.crop_size = 512 crop_size = opt.crop_size

I get these results, they vary on each run:

[Lyntai]

I managed to fix it (this may not be perfect and may have bugs), if anyone wants to try it, do this:

1. Download the checkpoints of the Simswap 512 beta version:

!wget https://github.com/neuralchen/SimSwap/releases/download/512_beta/512.zip
!unzip ./512.zip  -d ./checkpoints

2. You can swap with this code:

opt = TestOptions()
opt.initialize()
opt.parser.add_argument('-f') ## dummy arg to avoid bug
opt = opt.parse()
opt.pic_a_path = './demo_file/Iron_man.jpg' ## or replace it with image from your own google drive
opt.video_path = './demo_file/multi_people_1080p.mp4' ## or replace it with video from your own google drive
opt.output_path = './output/demo.mp4'
opt.temp_path = './tmp'
opt.Arc_path = './arcface_model/arcface_checkpoint.tar'
opt.isTrain = False
opt.use_mask = True  ## new feature up-to-date
opt.crop_size = 512
crop_size = opt.crop_size

torch.nn.Module.dump_patches = True
if crop_size == 512:
   opt.which_epoch = 550000
   opt.name = '512'
   mode = 'ffhq'
else:
   mode = 'None'

model = create_model(opt)
model.eval()

app = Face_detect_crop(name='antelope', root='./insightface_func/models')
app.prepare(ctx_id= 0, det_thresh=0.6, det_size=(640,640),mode=mode)

with torch.no_grad():
    pic_a = opt.pic_a_path
    # img_a = Image.open(pic_a).convert('RGB')
    img_a_whole = cv2.imread(pic_a)
    img_a_align_crop, _ = app.get(img_a_whole,crop_size)
    img_a_align_crop_pil = Image.fromarray(cv2.cvtColor(img_a_align_crop[0],cv2.COLOR_BGR2RGB)) 
    img_a = transformer_Arcface(img_a_align_crop_pil)
    img_id = img_a.view(-1, img_a.shape[0], img_a.shape[1], img_a.shape[2])

    # convert numpy to tensor
    img_id = img_id.cuda()

    #create latent id
    img_id_downsample = F.interpolate(img_id, size=(112,112))
    latend_id = model.netArc(img_id_downsample)
    latend_id = latend_id.detach().to('cpu')
    latend_id = latend_id/np.linalg.norm(latend_id,axis=1,keepdims=True)
    latend_id = latend_id.to('cuda')

    video_swap(opt.video_path, latend_id, model, app, opt.output_path, temp_results_dir=opt.temp_path, use_mask=opt.use_mask, crop_size=crop_size)

You can switch between 224 and 512 by modifying opt.crop_size value.

224

512

Congratulations to the authors for doing a wonderful job.

[NNNNAI]

Sorry for the late respond. Grate to see you make it work, I forget to update the colab file to the latest version yesterday, I will update those files today.Have a nice day.

[woctezuma]

I was contacted about this subject at https://github.com/woctezuma/SimSwap-colab/issues/3 and was a bit confused about the original issue, and where the fix was in the code posted above. So I will try to explain what I understood and see if I got it right after reading both of these issues:

• #162
• #198

As I understand, the issue only arises for people who use Colab notebooks instead of the official Python scripts. In these cases, the value of the crop_size variable is not set via the function argument --crop_size 512, but manually set by the user in the Colab notebook. This is an issue because opt.crop_size is later used by the program when create_model() is called, so the user is supposed to have set it to the non-default value (512) beforehand.

https://github.com/neuralchen/SimSwap/blob/050ed8a00a190aafafe183fdfa6a0eb845ded89d/models/fs_model.py#L53-L56

https://github.com/neuralchen/SimSwap/blob/dd1ecdd2a718636d33977ab3097a69a0ecf080d8/options/test_options.py#L36

The fix consists in setting opt.crop_size (to the same value as crop_size) in the Colab notebook.

[mayige]

How is this going?

[woctezuma]

I don't know which script you are trying to run, and you don't show the error message. I cannot help you with that. 😄

[Luby17]

I managed to fix it (this may not be perfect and may have bugs), if anyone wants to try it, do this:

1. Download the checkpoints of the Simswap 512 beta version:

!wget https://github.com/neuralchen/SimSwap/releases/download/512_beta/512.zip
!unzip ./512.zip  -d ./checkpoints

2. You can swap with this code:

opt = TestOptions()
opt.initialize()
opt.parser.add_argument('-f') ## dummy arg to avoid bug
opt = opt.parse()
opt.pic_a_path = './demo_file/Iron_man.jpg' ## or replace it with image from your own google drive
opt.video_path = './demo_file/multi_people_1080p.mp4' ## or replace it with video from your own google drive
opt.output_path = './output/demo.mp4'
opt.temp_path = './tmp'
opt.Arc_path = './arcface_model/arcface_checkpoint.tar'
opt.isTrain = False
opt.use_mask = True  ## new feature up-to-date
opt.crop_size = 512
crop_size = opt.crop_size

torch.nn.Module.dump_patches = True
if crop_size == 512:
   opt.which_epoch = 550000
   opt.name = '512'
   mode = 'ffhq'
else:
   mode = 'None'

model = create_model(opt)
model.eval()

app = Face_detect_crop(name='antelope', root='./insightface_func/models')
app.prepare(ctx_id= 0, det_thresh=0.6, det_size=(640,640),mode=mode)

with torch.no_grad():
    pic_a = opt.pic_a_path
    # img_a = Image.open(pic_a).convert('RGB')
    img_a_whole = cv2.imread(pic_a)
    img_a_align_crop, _ = app.get(img_a_whole,crop_size)
    img_a_align_crop_pil = Image.fromarray(cv2.cvtColor(img_a_align_crop[0],cv2.COLOR_BGR2RGB)) 
    img_a = transformer_Arcface(img_a_align_crop_pil)
    img_id = img_a.view(-1, img_a.shape[0], img_a.shape[1], img_a.shape[2])

    # convert numpy to tensor
    img_id = img_id.cuda()

    #create latent id
    img_id_downsample = F.interpolate(img_id, size=(112,112))
    latend_id = model.netArc(img_id_downsample)
    latend_id = latend_id.detach().to('cpu')
    latend_id = latend_id/np.linalg.norm(latend_id,axis=1,keepdims=True)
    latend_id = latend_id.to('cuda')

    video_swap(opt.video_path, latend_id, model, app, opt.output_path, temp_results_dir=opt.temp_path, use_mask=opt.use_mask, crop_size=crop_size)

You can switch between 224 and 512 by modifying opt.crop_size value.

224

512

Congratulations to the authors for doing a wonderful job.

I´ve copied the code and ended up with 256 anyway