BicycleGAN-pytorch "Expected more than 1 value per channel when training"

I'm testing this and trying to train edges2shoes model. When I run python train.py --root=data/edges2shoes --result_dir=result --weight_dir=weight I get ValueError: Expected more than 1 value per channel when training, got input size [1, 512, 1, 1]. I checked that the img and ground_truth are correctly shaped at [2, 3, 128, 128] so I'm not sure why it's doing this.

Any ideas?

Traceback (most recent call last):
  File "train.py", line 66, in <module>
    main(args)
  File "train.py", line 26, in main
    solver.train()
  File "C:\Users\foo\Development\BicycleGAN-pytorch\solver.py", line 211, in train
    fake_img_cVAE = self.G(cVAE_data['img'], encoded_z)
  File "C:\Users\foo\Miniconda3\envs\development\lib\site-packages\torch\nn\modules\module.py", line 357, in __call__
    result = self.forward(*input, **kwargs)
  File "C:\Users\foo\Development\BicycleGAN-pytorch\model.py", line 128, in forward
    down_7 = self.downsample_7(down_6)
  File "C:\Users\foo\Miniconda3\envs\development\lib\site-packages\torch\nn\modules\module.py", line 357, in __call__
    result = self.forward(*input, **kwargs)
  File "C:\Users\foo\Development\BicycleGAN-pytorch\model.py", line 38, in forward
    out = self.conv_block(x)
  File "C:\Users\foo\Miniconda3\envs\development\lib\site-packages\torch\nn\modules\module.py", line 357, in __call__
    result = self.forward(*input, **kwargs)
  File "C:\Users\foo\Miniconda3\envs\development\lib\site-packages\torch\nn\modules\container.py", line 67, in forward
    input = module(input)
  File "C:\Users\foo\Miniconda3\envs\development\lib\site-packages\torch\nn\modules\module.py", line 357, in __call__
    result = self.forward(*input, **kwargs)
  File "C:\Users\foo\Miniconda3\envs\development\lib\site-packages\torch\nn\modules\instancenorm.py", line 28, in forward
    not self.use_running_stats, self.momentum, self.eps)
  File "C:\Users\foo\Miniconda3\envs\development\lib\site-packages\torch\nn\functional.py", line 1011, in batch_norm
    raise ValueError('Expected more than 1 value per channel when training, got input size {}'.format(size))
ValueError: Expected more than 1 value per channel when training, got input size [1, 512, 1, 1]