Custom Weight Initialization

[glenn-jocher]

I noticed you use code for custom weight initialization: https://github.com/iamhankai/ghostnet.pytorch/blob/2c90e67d8c33c44ec1bad12c9686f645b0d4de08/ghost_net.py#L162-L169

I've not seen this before. Is there a reason behind this specific strategy? Do you know the effect this has on the training, and have you compared this with the pytorch default weight initialization? Thank you!

[iamhankai]

kaiming_normal_ is a commonly used initialization strategy.

[glenn-jocher]

@iamhankai thank you! Do you know what the default pytorch weights init strategy is?

I suppose this makes for easier comparisons with the TF version of ghostnet to use the same strategy on both?

[iamhankai]

@glenn-jocher TF version of ghostnet also used Kaiming normal initialization.