KernelGAN icon indicating copy to clipboard operation
KernelGAN copied to clipboard

why not directly save the params of Generator for downscaling? why not non-linear?

Open hitsz-zuoqi opened this issue 3 years ago • 3 comments

As shown in paper, the generator is equaled to a simple linear kernel, then why we still need to use 'imresize' func to resize the picture, can we save all weights of generator for downscaling?

Another question, I see the post_process_k func and I think even a non-linear generator can be postprocessed as a fine filter, and also the non-linear filter can be computed by the same way.

Could you please explain about the two questions?

hitsz-zuoqi avatar Jul 27 '21 08:07 hitsz-zuoqi

As shown in paper, the generator is equaled to a simple linear kernel, then why we still need to use 'imresize' func to resize the picture, can we save all weights of generator for downscaling?

Yes. But that will be computationally more expensive. Even simply the first layer converting to 64 channels is much more expensive

Another question, I see the post_process_k func and I think even a non-linear generator can be postprocessed as a fine filter, and also the non-linear filter can be computed by the same way.

A non-linear filter would be able to "overfit" and produce random patches of the image regardless of the input. That way we would not be learning a downscaling function. Additionally, what does a non-linear kernel relate to in real-life?

sefibk avatar Jul 27 '21 08:07 sefibk

I see the multiplicative noise experiments in paper, maybe the kernel after multiplication is non-linear? In real-life, non-linearity can also be caused by image compression or other factors?

hitsz-zuoqi avatar Jul 27 '21 08:07 hitsz-zuoqi

Definitely can, the problem is I am not sure of how to enforce the Generator to learn only "realistic" models. As I mentioned - it can easily "memorize" the patches of the image and output only those patches and fool the discriminator. There is actually a great work that does exactly that (InGAN)

sefibk avatar Jul 27 '21 16:07 sefibk