few-shot-vid2vid
few-shot-vid2vid copied to clipboard
[Pose] A problem during testing
In the "Few-shot Video-to-Video Synthesis":
Moreover, we can also (optionally) finetune the network using the given example images to improve performance.Note that we only finetune the weight generation module E and the intermediate image synthesis network H, and leave all parameters related to flow estimation fixed.
But in the code:
train_names = ['fc', 'conv_img', 'up']
params, _ = self.get_train_params(self.netG, train_names)
and
def get_train_params(self, netG, train_names):
train_list = set()
params = []
params_dict = netG.state_dict()
for key, value in params_dict.items():
do_train = False
for model_name in train_names:
if model_name in key: do_train = True
if do_train:
module = netG
key_list = key.split('.')
for k in key_list:
module = getattr(module, k)
params += [module]
train_list.add('.'.join(key_list[:1]))
Visualizer.vis_print(self.opt, ('training layers: ', train_list))
return params, train_list
During testing, I get the output:
('training layers: ', {'fc_spade_0_0', 'ref_img_up_2', 'fc_spade_e_0', 'fc_spade_1_3', 'conv_img', 'fc_spade_1_0', 'ref_img_up_4', 'label_embedding', 'fc_spade_s_0', 'fc_spade_1_1', 'ref_label_up_1', 'fc_spade_0_3', 'fc_spade_e_3', 'fc_spade_s_3', 'fc_spade_e_2', 'up_4', 'up_2', 'ref_label_up_2', 'up_1', 'ref_label_up_3', 'ref_img_up_0', 'up_0', 'fc_spade_1_2', 'fc_spade_s_1', 'fc_spade_s_2', 'ref_label_up_0', 'ref_label_up_4', 'flow_network_temp', 'fc_spade_0_2', 'flow_network_ref', 'ref_img_up_1', 'up_5', 'fc_spade_0_1', 'up_3', 'ref_img_up_3', 'fc_spade_e_1'})
In fact, the "flow_network_ref" and "flow_network_temp" are also finetuned. The result is completely collapsed.
Have you solved this problem? I also met the same problem.
I filter out the optical flow network and tune down my learning rate. After that, the results seem to be reasonable but still unsatisfactory.
I filter out the optical flow network and tune down my learning rate. After that, the results seem to be reasonable but still unsatisfactory.
I used the same method as you, and the results were not satisfactory,especially in the facial area. In addition, did you set the "--add_face_D" option? After I use this option to train for a while, the synthesised facial area will collapse.
Sorry for the late replay. I didn't set the "--add_face_D" option.