Self-Attention-GAN-Tensorflow icon indicating copy to clipboard operation
Self-Attention-GAN-Tensorflow copied to clipboard

Published 20 hours ago • verified publisher icon taki0112

About SN & BN

Open r06922019 opened this issue 6 years ago • 0 comments

Hi,

Great work! I’m wondering if applying BN with scale=True make the network no longer be a Lipschitz-1 function (Which should be the target of SN?)

In the paper of SN, we treat the entire network as a function and calculate its Lipschitz constant by multiplying every Lipschitz constant of each component And since BN with scale=True introduces an additional scaling parameter, we should also take it into consideration when calculating its spectral norm, right? If so, applying BN after SN seems to destroy the work done by SN?

r06922019 avatar Jan 19 '19 19:01 r06922019