Missing abs() when calculating the inter-filter orthogonality?

[qinjian623]

https://github.com/siahuat0727/RePr/blob/4fa1b8d30da1d9f88ccc29fe3ab1711811ef093d/main.py#L148

In paper it seems need an abs() op here.

[siahuat0727]

Oh! You are right, I made a terrible mistake here! The result without abs() can't represent its orthogonality among others. I will fix it and re-run the experiments then. Thanks a lot!

[qinjian623]

Hi, I've ran the experiment with abs op. But found out the ranking prefer to prune the shallow layer of net. Maybe there are mistakes in my experiment settings.

After your rerun-experiment, would you mind to info me about your experiment result please?

Thanks.

[siahuat0727]

Sorry for the late reply.

My results are the same as yours. Maybe there are more bugs in my implementation. I also hope you can let me know when you find other problems. Thanks.

[qinjian623]

Hi,

I have my own implementation. It also prefers shallow layer. Maybe we need contact the paper's author for more details.

Seems you speak Chinese. I think we can keep in touch with e-mail.

[siahuat0727]

No problem. Keep in touch!

[BigFishMaster]

Have you re-run your experiments? @siahuat0727

[BigFishMaster]

@qinjian623 I have contacted you at weibo, please keep on touching.

[siahuat0727]

@BigFishMaster Hi, I will re-run it this weekend and update the results on my blog.

[BigFishMaster]

@siahuat0727 This is the discussion on reddit. Somebodys doubt the result. https://www.reddit.com/r/MachineLearning/comments/ayh2hf/r_repr_improved_training_of_convolutional_filters/

[BigFishMaster]

@siahuat0727 the result is too high in Fig.1

[BigFishMaster]

@siahuat0727 orthogonal function in pytorch may help. https://pytorch.org/docs/stable/nn.html?highlight=orthogonal#torch.nn.init.orthogonal_

[siahuat0727]

@BigFishMaster Thanks! I've just updated my re-run results.