Adashift

Reproducing of the AdaShift: Decorrelation and Convergence of Adaptive Learning Rate Methods as a part of ICLR Reproducibility Challange 2019. See our report.

Experiments

Synthetic Experiment

Logistic Regression on MNIST

W-GAN

NMT

Dependencies

• Python 3.4+,
• NumPy, Pandas
• PyTorch 0.4+
• SciPy, Matplotlib
• OpenNMT-py for NMT
• GPU for DL experiments

References

On the Convergence of Adam and Beyond

Adam

AdaShift: Decorrelation and Convergence of Adaptive Learning Rate Methods