LTR-weight-balancing icon indicating copy to clipboard operation
LTR-weight-balancing copied to clipboard

Published 20 hours ago • verified publisher icon ShadeAlsha

Questions about Maxnorm regularizations

Open chenbinghui1 opened this issue 3 years ago • 4 comments

In paper, it claims that Maxnorm regularization is applied per iteration, but in the code, it seems to be applied per epoch? Am i right?

chenbinghui1 avatar Jun 28 '22 07:06 chenbinghui1

In the demo, MaxNorm is applied in each epoch as you said. This corresponds to lines 140-142 in trainval.py.

https://github.com/ShadeAlsha/LTR-weight-balancing/blob/0e9494cad5b4805642f05097a336096f780300ee/utils/trainval.py#L140-L142

You can move this code up in order to apply it per-iteration as in the paper. I will update the shared open-source code soon. Thank you!

ShadeAlsha avatar Jun 28 '22 17:06 ShadeAlsha

Looks like there is a significant difference in performance between the two. Resnet34. Benchmarked with the code provided.

Screenshot 2022-07-01 at 9 35 29 AM

@ShadeAlsha @chenbinghui1

rahulvigneswaran avatar Jul 01 '22 04:07 rahulvigneswaran

Yes, PGD is supposed to be applied in each iteration as in page 4 of the paper and as chenbinghui1 mentioned earlier :) I haven't updated the code yet, but in the mean time your modified version should be good if you just moved PGD to the iteration loop. Thanks for your interest!

ShadeAlsha avatar Jul 01 '22 05:07 ShadeAlsha

Hello, I would like to ask, when I used the author's source code to fine-tune, how is it that the addition and removal of maximum regularization are the same? Does it seem to not work?

xiaohe6 avatar Nov 10 '22 08:11 xiaohe6