mxnet
mxnet copied to clipboard
[FEATURE] RAdam optimizer implementation
Description
RAdam optimizer Python implementation as requested in #9182
Checklist
Essentials
- [X] PR's title starts with a category (e.g. [BUGFIX], [MODEL], [TUTORIAL], [FEATURE], [DOC], etc)
- [X] Changes are complete (i.e. I finished coding on this PR)
- [X] All changes have test coverage
- [X] Code is well-documented
Changes
- [X] RAdam optimizer with internal python doc.
- [X] Basic test added to the appropriate python file.
Comments
- Default argument values
beta1, beta2, epsilonare aligned with the current PyTorch implementation and both the Tensorflow and PyTorch implementations were referenced for the implementation. - Due to hardware restrictions I have not been able to run a proper test of the optimizer so if I could get some help with that, it would be appreciated.
Hey @Hunter-Zolomon , Thanks for submitting the PR All tests are already queued to run once. If tests fail, you can trigger one or more tests again with the following commands:
- To trigger all jobs: @mxnet-bot run ci [all]
- To trigger specific jobs: @mxnet-bot run ci [job1, job2]
CI supported jobs: [clang, windows-cpu, website, edge, miscellaneous, centos-gpu, unix-gpu, sanity, windows-gpu, centos-cpu, unix-cpu]
Note: Only following 3 categories can trigger CI :PR Author, MXNet Committer, Jenkins Admin. All CI tests must pass before the PR can be merged.