OctoberKat

I wonder if I can load the model parameters to GPU correctly by simply write: ``` model = EfficientNet.from_pretrained('efficientnet-b3') model.cuda() ```

Maybe you should try warmup learning rate sceduler? Transformer is particularly sensitive to learning rate scheme.