skglm
skglm copied to clipboard
FEAT - Add alpha_max methods to estimators
Description of the feature
In multiple issues, it seems that practitioners had problems selecting the regularization parameter alpha, and a lot of them do not know the value of alpha_max.
Maybe we could implement an alpha_max method for popular estimators.
We have them for some penalties : https://github.com/scikit-learn-contrib/skglm/blob/main/skglm/penalties/separable.py#L206
It regularly pops up because people are not aware of this and use an alpha_max that way too small. I don't know how to fix this in a unified fashion (it does not apply to L2 regularized problems for example). I think having an example to point to, at least, would make answering the same question repeatedly easier.
From a user point of view, I would see a function estimator.get_alpha_max(X, y), with
def get_alpha_max(X, y):
return self.penalty.dual(X.T @ self.fdatafit.rawgrad(y, 0))
To me, this is unclear how to do it without repeating this chunk of code