peiwang062

In fact, the detailed explanations are following this sentence. This is one experimental observation. It is like 'whack-a-mole' game. It is hard to generate large and negative outputs of BN...

This is a great question. I guess using leakyrelu should work as well. We didn't talk about this possibility is that because our motivation is not to propose a new...

In our experiments, we found that there is no big difference for vgg with or without normalization, So we didn't explore too much on this line. And for resnet, if...

Hi, these codes haven't been polished yet. Will upload then. The idea is the same as that of ResNet, just adding smoothing before the computation of the loss. The implementation...

Equations 5-8 describe the calculation. In fact, no difference with the basic entropy in the textbook, we just normalize to [0,1], this is also a standard operation.

For 1, first of all, everything discussed in the paper is in the scope of those methods by L2 loss and Gram matrix based optimization (the very popular and common...

In my experiments, I mainly did on random network. The observation is deeper layers have lower entropy, say resnet. For well trained models, the same observation is but the discrepancy...

yes, build-in. something like this from scipy.stats import entropy feature = featuremap.cpu().data.numpy().squeeze() feature_stat = feature.flatten() feature_stat = softmax(feature_stat) cur_entropy = entropy(feature_stat) / np.log(len(feature_stat.tolist()))

For different style images, the entropy is difference. The shown in our plots is the average.

I saw you are using VGG19, for VGG, entropy doesn't decrease. It is for resnet.