SVRG and LSVRG approximate gradient methods
Describe your changes
SVRG and LSVRG approximate gradient methods
Implements: SVRG
- https://proceedings.neurips.cc/paper/2013/file/ac1dd209cbcc5e5d1c6e28598e8cbbe8-Paper.pdf
- https://github.com/epapoutsellis/StochasticCIL/blob/svrg/Wrappers/Python/cil/optimisation/functions/SVRGFunction.py
LSVRG
- https://github.com/epapoutsellis/StochasticCIL/blob/svrg/Wrappers/Python/cil/optimisation/functions/SVRGFunction.py
Describe any testing you have performed
Please add any demo scripts to CIL-Demos/misc/
Link relevant issues
Closes #1623
Checklist when you are ready to request a review
- [x] I have performed a self-review of my code
- [x] I have added docstrings in line with the guidance in the developer guide
- [x] I have implemented unit tests that cover any new or modified functionality
- [x] CHANGELOG.md has been updated with any functionality change
- [x] Request review from all relevant developers
- [x] Change pull request label to 'Waiting for review'
Contribution Notes
Please read and adhere to the developer guide and local patterns and conventions.
- [x] The content of this Pull Request (the Contribution) is intentionally submitted for inclusion in CIL (the Work) under the terms and conditions of the Apache-2.0 License.
- [x] I confirm that the contribution does not violate any intellectual property rights of third parties
I was starting to look at this and see it contains some of the same code that is provided by #1550, on which changes are currently being implemented. What is the process for getting these synced and merged? Should the present PR be merged into #1550 instead of main? Or #1550 into the present one? I suppose I can comment on just the files that are NOT provided by #1550?
I was starting to look at this and see it contains some of the same code that is provided by #1550, on which changes are currently being implemented. What is the process for getting these synced and merged? Should the present PR be merged into #1550 instead of main? Or #1550 into the present one? I suppose I can comment on just the files that are NOT provided by #1550?
Thanks @jakobsj - The plan is to merge #1550 with main and then merge main into this one. Yes please, comments on the implementation in the SVRGFunction.py file
Jenkins is happy