LossFunctions.jl icon indicating copy to clipboard operation
LossFunctions.jl copied to clipboard

Lipschitz constants for Lipschitz losses ?

Open stephanegaiffas opened this issue 3 years ago • 1 comments

Hi all and thanks for the this very useful package ! It could be helpful to get in addition to the islipschitzcont attribute, another attribute that gives the value of the global Lipschitz constant, e.g. 2 for your scaling of the L2 loss, 1/4 for the logistic loss, etc. If I am not wrong, this is not featured yet in the package ?

This could be particularly useful for some first-order optimization algorithms in ML, where theoretically optimal learning rates can be automatically tuned using Lipschitz constants (and strong convexity constants as well).

If this is welcomed I could try a PR that implements this.

Best !

stephanegaiffas avatar Aug 15 '21 14:08 stephanegaiffas

@stephanegaiffas can you draft a PR with a proposal? We will be happy to review it.

juliohm avatar May 22 '22 14:05 juliohm