Pangu lite training

[mikael10j]

Hello,

I'm trying to reproduce the Pangu Lite results shown in the README. However, although I followed the pseudocode and the paper for the implementation, I don't get the same RMSEs (I'm quite far actually).

Could you please confirm if you used the same hyperparameters as for the full version: optimizer = Adam lr = 5e-4 weight decay = 3e-6 lr scheduler = cosine

Thank you