Srgan seems to only support scale=4

[bluelight773]

I'm trying to train an Srgan per example-srgan.ipynb just using scale=8 instead of scale=4. However, as soon as I try to run:

pre_trainer.train(train_ds,
                  valid_ds.take(10),
                  steps=1000000, 
                  evaluate_every=1000, 
                  save_best_only=False)

I get an error

ValueError: Dimensions must be equal, but are 48 and 96 for '{{node mean_squared_error/SquaredDifference}} = SquaredDifference[T=DT_FLOAT](functional_1/lambda_3/mul, Cast_1)' with input shapes: [16,48,48,3], [16,96,96,3].

I found that I also get similar errors when I try a scale value other than 4, such as scale=2.

Is there something I can do to make Srgan training including finetuning work with scale=8?

[GlockPL]

I managed to find a way to solve this. You need to comment Line 42 in file srgan.py: x = upsample(x, num_filters * 4). One of the upscaling layer, this way it will only upscale once by factor 2.

[yasho191]

If you take a closer look at the code you will understand that currently, there are only 2 hard coded upsampling layers which means a scaling factor of 4 you need to increase or decrease those according to your scaling factor. So if you want to train it for scale = x8 you need 3 upsampling layers.

Refer Line 41 and 42 of srgan.py

# scale = x2
x = upsample(x, num_filters * 4)

# scale = x4
x = upsample(x, num_filters * 4)
x = upsample(x, num_filters * 4)

# for scale = x8
x = upsample(x, num_filters * 4)
x = upsample(x, num_filters * 4)
x = upsample(x, num_filters * 4)

# or
for i in range(3):
    x = upsample(x, num_filters * 4)

[GruNyv]

Do you know if it is possible to upsample along one direction only, for instance only the vertical direction?