pytorch-optimizer
pytorch-optimizer copied to clipboard
jettify
torch-optimizer -- collection of optimizers for Pytorch
Hi, I was using the SGDW implementation in this repo, and I wonder if anything is wrong with this line: https://github.com/jettify/pytorch-optimizer/blob/910b414565427f0a66e20040475e7e4385e066a5/torch_optimizer/sgdw.py#L121 Let weight decay be $\lambda$ and learning rate be...
Hi, I open this pull request for the issue I submitted regarding SGDW: https://github.com/jettify/pytorch-optimizer/issues/454#issue-1332395921 Hope this change makes sense. Thanks! Best Regards, Liu
Bumps [torch](https://github.com/pytorch/pytorch) from 1.10.0 to 1.12.1. Release notes Sourced from torch's releases. PyTorch 1.12.1 Release, small bug fix release This release is meant to fix the following issues (regressions /...
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot] avatar"){kind=avatar}
The documentation suggests that the Shampoo algorithm comes from the 2018 paper: https://arxiv.org/abs/1802.09568 Is this correct? If so, are there any plan to upgrade to the authors' newer 2021 paper:...
Bumps [sphinx](https://github.com/sphinx-doc/sphinx) from 4.2.0 to 5.1.1. Release notes Sourced from sphinx's releases. v5.1.1 Changelog: https://www.sphinx-doc.org/en/master/changes.html v5.1.0 Changelog: https://www.sphinx-doc.org/en/master/changes.html v5.0.2 Changelog: https://www.sphinx-doc.org/en/master/changes.html v5.0.1 Changelog: https://www.sphinx-doc.org/en/master/changes.html v5.0.0 No release notes provided. v5.0.0b1...
Bumps [numpy](https://github.com/numpy/numpy) from 1.21.3 to 1.23.1. Release notes Sourced from numpy's releases. v1.23.1 NumPy 1.23.1 Release Notes The NumPy 1.23.1 is a maintenance release that fixes bugs discovered after the...
Bumps [flake8-bugbear](https://github.com/PyCQA/flake8-bugbear) from 21.9.2 to 22.7.1. Release notes Sourced from flake8-bugbear's releases. 22.7.1 Implement late-binding loop check (#265) late-binding closures are a classic gotcha. 22.6.22 Don't crash when select /...
Bumps [black](https://github.com/psf/black) from 21.9b0 to 22.6.0. Release notes Sourced from black's releases. 22.6.0 Style Fix unstable formatting involving #fmt: skip and # fmt:skip comments (notice the lack of spaces) (#2970)...
Hi, I was checking your lamb implementation and I think there is a mistake in it. According to the paper, exp_avg and exp_avg_sq (m and v) must be updated this...
Bumps [twine](https://github.com/pypa/twine) from 3.4.2 to 4.0.1. Release notes Sourced from twine's releases. 4.0.1 https://pypi.org/project/twine/4.0.1/ Changelog 4.0.0 https://pypi.org/project/twine/4.0.0/ Changelog 3.8.0 https://pypi.org/project/twine/3.8.0/ Changelog 3.7.1 https://pypi.org/project/twine/3.7.1/ Changelog 3.7.0 https://pypi.org/project/twine/3.7.0/ Changelog 3.6.0 https://pypi.org/project/twine/3.6.0/ Changelog...