august779188

hi,could you tell me what args should be selected in the pretraining process,"resume or loadckpt". do i need to load the state of the optimizers in the relu stage?thank you

I also found this problem ：the gradients of gamma_s3, gamma_s2, beta_s3, beta_s2 are None ，and they are not updated during training.it will lead to poor training results