skglm icon indicating copy to clipboard operation
skglm copied to clipboard

FEAT - Add alpha_max methods to estimators

Open QB3 opened this issue 1 year ago • 2 comments

Description of the feature

In multiple issues, it seems that practitioners had problems selecting the regularization parameter alpha, and a lot of them do not know the value of alpha_max. Maybe we could implement an alpha_max method for popular estimators.

QB3 avatar Apr 03 '24 19:04 QB3

We have them for some penalties : https://github.com/scikit-learn-contrib/skglm/blob/main/skglm/penalties/separable.py#L206

It regularly pops up because people are not aware of this and use an alpha_max that way too small. I don't know how to fix this in a unified fashion (it does not apply to L2 regularized problems for example). I think having an example to point to, at least, would make answering the same question repeatedly easier.

mathurinm avatar May 31 '24 11:05 mathurinm

From a user point of view, I would see a function estimator.get_alpha_max(X, y), with

def get_alpha_max(X, y):
      return self.penalty.dual(X.T @ self.fdatafit.rawgrad(y, 0))

To me, this is unclear how to do it without repeating this chunk of code

QB3 avatar Jun 03 '24 14:06 QB3