L-BFGS in the algorithm class?

[MargaretDuff]

Description

I have had a few discussions recently about algorithms with some second order approximation to improve convergence speed.

Something like L-BFGS could be a start https://en.wikipedia.org/wiki/Limited-memory_BFGS.

This issue could be included in a wider discussion about priorities for the algorithm class in CIL.

Environment

import cil, sys
print(cil.version.version, cil.version.commit_hash, sys.version, sys.platform)

[MargaretDuff]

Also used by @bosschmidt in the user meeting. We were comparing his results using scipy optimise and with code implemented in CIL. BFGS in scipy converged quickly even with his ill-conditioned problem