LossFunctions.jl
LossFunctions.jl copied to clipboard
Lipschitz constants for Lipschitz losses ?
Hi all and thanks for the this very useful package ! It could be helpful to get in addition to the islipschitzcont attribute, another attribute that gives the value of the global Lipschitz constant, e.g. 2 for your scaling of the L2 loss, 1/4 for the logistic loss, etc. If I am not wrong, this is not featured yet in the package ?
This could be particularly useful for some first-order optimization algorithms in ML, where theoretically optimal learning rates can be automatically tuned using Lipschitz constants (and strong convexity constants as well).
If this is welcomed I could try a PR that implements this.
Best !
@stephanegaiffas can you draft a PR with a proposal? We will be happy to review it.