slala2121

Converting from tensorflow to pytorch

4

Platform (like ubuntu 16.04/win10): ubuntu 16.04 Python version: 3 Source framework with version (like Tensorflow 1.4.1 with GPU): Tensorflow 1.15.0 Destination framework with version (like CNTK 2.3 with GPU): Pytorch...

Spikes in training loss

1

I'm wondering if you have encountered this recurring spike in the training loss and why might this occur. It does appear to occur upon processing the last batch but I...

Loss, entropy, accuracy trends

5

I'm trying to understand the relationships and trends among these quantities. From some experiments, I find that 1. as the training loss declines, the entropy of the distribution increases rather...

Selective kernel ratio hyperparameter

2

For the SSL pretrained and supervised trained ImageNet models, what was the sk_ratio? Also, was the se_ratio = 0? What does the se_ratio represent? Thanks.

Are there sample training loss plots available for the contrastive learning studies e.g., imagenet?

drange of the generative model

1

Does the drange affect the quality of the generated images? I think the default is [-1,1] in the code. Is there a reason why this as opposed to [0,1]? e.g.,...

BatchNorm behavior during inference

1

This is bit more detailed but I wanted to check if the batch norm layer of the computation graph (1st picture below) matches the one you find when importing the...

Hi, Minor issue but seems like the lr scheduler step should be called after the training and discriminator training steps (pytorch 1.1.0+) else the first lr is skipped.