pytorch-CycleGAN-and-pix2pix icon indicating copy to clipboard operation
pytorch-CycleGAN-and-pix2pix copied to clipboard

Trained a cyclegann model, Unexpected key error on testing it

Open thijser opened this issue 4 years ago • 3 comments

I have trained a model with the following options:


----------------- Options ---------------
               batch_size: 1                             
                    beta1: 0.5                           
          checkpoints_dir: ./checkpoints                 
           continue_train: False                         
                crop_size: 256                           
                 dataroot: ./datasets/datasets/boundedwalls_50_0.1/	[default: None]
             dataset_mode: aligned                       	[default: unaligned]
                direction: AtoB                          
              display_env: main                          
             display_freq: 400                           
               display_id: 1                             
            display_ncols: 4                             
             display_port: 8097                          
           display_server: http://localhost              
          display_winsize: 256                           
                    epoch: latest                        
              epoch_count: 1                             
                 gan_mode: lsgan                         
                  gpu_ids: 0                             
                init_gain: 0.02                          
                init_type: normal                        
                 input_nc: 3                             
                  isTrain: True                          	[default: None]
                 lambda_A: 10.0                          
                 lambda_B: 10.0                          
          lambda_identity: 0.5                           
                load_iter: 0                             	[default: 0]
                load_size: 286                           
                       lr: 0.0002                        
           lr_decay_iters: 50                            
                lr_policy: linear                        
         max_dataset_size: inf                           
                    model: cycle_gan                     
                 n_epochs: 100                           
           n_epochs_decay: 100                           
               n_layers_D: 3                             
                     name: cycbw50                       	[default: experiment_name]
                      ndf: 64                            
                     netD: basic                         
                     netG: resnet_9blocks                
                      ngf: 64                            
               no_dropout: True                          
                  no_flip: False                         
                  no_html: False                         
                     norm: batch                         	[default: instance]
              num_threads: 4                             
                output_nc: 3                             
                    phase: train                         
                pool_size: 50                            
               preprocess: resize_and_crop               
               print_freq: 100                           
             save_by_iter: False                         
          save_epoch_freq: 5                             
         save_latest_freq: 5000                          
           serial_batches: False                         
                   suffix:                               
         update_html_freq: 1000                          
                  verbose: False                         
----------------- End -------------------

And would like to test it, the test has the following options right now (correct me if you see what's wrong)

----------------- Options ---------------
             aspect_ratio: 1.0                           
               batch_size: 1                             
          checkpoints_dir: ./checkpoints                 
                crop_size: 256                           
                 dataroot: ./datasets/datasets/boundedwalls_50_0.1/	[default: None]
             dataset_mode: unaligned                     
                direction: AtoB                          
          display_winsize: 256                           
                    epoch: latest                        
                     eval: False                         
                  gpu_ids: 0                             
                init_gain: 0.02                          
                init_type: normal                        
                 input_nc: 3                             
                  isTrain: False                         	[default: None]
                load_iter: 0                             	[default: 0]
                load_size: 256                           
         max_dataset_size: inf                           
                    model: cycle_gan                     	[default: test]
               n_layers_D: 3                             
                     name: cycbw50                       	[default: experiment_name]
                      ndf: 64                            
                     netD: basic                         
                     netG: resnet_9blocks                
                      ngf: 64                            
               no_dropout: True                          
                  no_flip: False                         
                     norm: instance                      
                 num_test: 50                            
              num_threads: 4                             
                output_nc: 3                             
                    phase: test                          
               preprocess: resize_and_crop               
              results_dir: ./results/                    
           serial_batches: False                         
                   suffix:                               
                  verbose: False                         
----------------- End -------------------

When I run the test however I get the error:

model [CycleGANModel] was created
loading the model from ./checkpoints/cycbw50/latest_net_G_A.pth
Traceback (most recent call last):
  File "test.py", line 47, in <module>
    model.setup(opt)               # regular setup: load and print networks; create schedulers
  File "/media/bitlockermount/SmartImageToDigitalTwin/SmartImageToDigitalTwin/bin/python/cyclegann/pytorch-CycleGAN-and-pix2pix/models/base_model.py", line 88, in setup
    self.load_networks(load_suffix)
  File "/media/bitlockermount/SmartImageToDigitalTwin/SmartImageToDigitalTwin/bin/python/cyclegann/pytorch-CycleGAN-and-pix2pix/models/base_model.py", line 199, in load_networks
    net.load_state_dict(state_dict)
  File "/home/bst/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 846, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for ResnetGenerator:
	Missing key(s) in state_dict: "model.1.bias", "model.4.bias", "model.7.bias", "model.10.conv_block.1.bias", "model.10.conv_block.5.bias", "model.11.conv_block.1.bias", "model.11.conv_block.5.bias", "model.12.conv_block.1.bias", "model.12.conv_block.5.bias", "model.13.conv_block.1.bias", "model.13.conv_block.5.bias", "model.14.conv_block.1.bias", "model.14.conv_block.5.bias", "model.15.conv_block.1.bias", "model.15.conv_block.5.bias", "model.16.conv_block.1.bias", "model.16.conv_block.5.bias", "model.17.conv_block.1.bias", "model.17.conv_block.5.bias", "model.18.conv_block.1.bias", "model.18.conv_block.5.bias", "model.19.bias", "model.22.bias". 
	Unexpected key(s) in state_dict: "model.2.weight", "model.2.bias", "model.5.weight", "model.5.bias", "model.8.weight", "model.8.bias", "model.10.conv_block.2.weight", "model.10.conv_block.2.bias", "model.10.conv_block.6.weight", "model.10.conv_block.6.bias", "model.11.conv_block.2.weight", "model.11.conv_block.2.bias", "model.11.conv_block.6.weight", "model.11.conv_block.6.bias", "model.12.conv_block.2.weight", "model.12.conv_block.2.bias", "model.12.conv_block.6.weight", "model.12.conv_block.6.bias", "model.13.conv_block.2.weight", "model.13.conv_block.2.bias", "model.13.conv_block.6.weight", "model.13.conv_block.6.bias", "model.14.conv_block.2.weight", "model.14.conv_block.2.bias", "model.14.conv_block.6.weight", "model.14.conv_block.6.bias", "model.15.conv_block.2.weight", "model.15.conv_block.2.bias", "model.15.conv_block.6.weight", "model.15.conv_block.6.bias", "model.16.conv_block.2.weight", "model.16.conv_block.2.bias", "model.16.conv_block.6.weight", "model.16.conv_block.6.bias", "model.17.conv_block.2.weight", "model.17.conv_block.2.bias", "model.17.conv_block.6.weight", "model.17.conv_block.6.bias", "model.18.conv_block.2.weight", "model.18.conv_block.2.bias", "model.18.conv_block.6.weight", "model.18.conv_block.6.bias", "model.20.weight", "model.20.bias", "model.23.weight", "model.23.bias". 

Do you see what I'm doing wrong here or is there some kind of bug in the code?

test_opt.txt train_opt.txt

thijser avatar May 05 '21 09:05 thijser

Hello, I am facing similar issue. Were you able to solve?

mahmed1995 avatar Jul 19 '21 10:07 mahmed1995

Hey,

I think this is due to you specified the norm=batch in your train_opt.txt. In the code, if you apply the batchNom instead of the instanceNorm, then the bias in each conv2d layer will be removed (check here and here for details). So your trained weights actually store no bias at all, that's why Missing key(s) errors were raised.

I think your test option should follow the same setting, i.e., specifying the norm=batch. Hope this helps.

LiUzHiAn avatar Nov 10 '21 05:11 LiUzHiAn

Thanks, @LiUzHiAn . Also see this FAQ for more details.

junyanz avatar Dec 02 '21 21:12 junyanz