JuliaSmoothOptimizers
Regularized Quasi-Newton operators
It would be useful to have an implementation of the operator $$v\mapsto Hv + \sigma v$$, where $$H$$ is some quasi-Newton operator and $$\sigma > 0$$. We should be able to get the inverse of this operator cheaply, see https://arxiv.org/pdf/1911.04584 .
cc @MohamedLaghdafHABIBOULLAH, i don't know if you plan to implement this soon ? I might do it on my own as well, I will need it for my research.
If you need it immediately, feel free to open a PR, and we can review it together.
Indeed, it performs the inverse operation, but only for LBFGSOperator.
@farhadrclass Already implemented the exact solve of $$(B + \sigma I) x = b$$ for LBFGS. It can be implemented for other limited-memory quasi-Newton operators but I believe we need to implement the compact storage first (there was an attempt in #95).
Useful references:
- https://www.sciencedirect.com/science/article/pii/S0024379512001358
- https://onlinelibrary.wiley.com/doi/full/10.1002/nla.2186
- https://ieeexplore.ieee.org/abstract/document/6810448
- https://www.tandfonline.com/doi/full/10.1080/10556788.2014.894045
Note that, more generally, it’s possible to solve $(B + G) x = b$ where $G$ is symmetric provided the user supplies a function to perform a solve with $G$. That would be a great contribution.
Interesting @dpo! That will be useful for the Becker/Kanzow proxes as well.