mxnet icon indicating copy to clipboard operation
mxnet copied to clipboard

[FEATURE] RAdam optimizer implementation

Open Hunter-Zolomon opened this issue 4 years ago • 1 comments

Description

RAdam optimizer Python implementation as requested in #9182

Checklist

Essentials

  • [X] PR's title starts with a category (e.g. [BUGFIX], [MODEL], [TUTORIAL], [FEATURE], [DOC], etc)
  • [X] Changes are complete (i.e. I finished coding on this PR)
  • [X] All changes have test coverage
  • [X] Code is well-documented

Changes

  • [X] RAdam optimizer with internal python doc.
  • [X] Basic test added to the appropriate python file.

Comments

  • Default argument values beta1, beta2, epsilon are aligned with the current PyTorch implementation and both the Tensorflow and PyTorch implementations were referenced for the implementation.
  • Due to hardware restrictions I have not been able to run a proper test of the optimizer so if I could get some help with that, it would be appreciated.

Hunter-Zolomon avatar Nov 29 '21 14:11 Hunter-Zolomon

Hey @Hunter-Zolomon , Thanks for submitting the PR All tests are already queued to run once. If tests fail, you can trigger one or more tests again with the following commands:

  • To trigger all jobs: @mxnet-bot run ci [all]
  • To trigger specific jobs: @mxnet-bot run ci [job1, job2]

CI supported jobs: [clang, windows-cpu, website, edge, miscellaneous, centos-gpu, unix-gpu, sanity, windows-gpu, centos-cpu, unix-cpu]


Note: Only following 3 categories can trigger CI :PR Author, MXNet Committer, Jenkins Admin. All CI tests must pass before the PR can be merged.

mxnet-bot avatar Nov 29 '21 14:11 mxnet-bot