mlr3learners icon indicating copy to clipboard operation
mlr3learners copied to clipboard

xgboost early stopping

Open sebffischer opened this issue 1 year ago • 3 comments

Because I just reviewed a PR in extralearners (https://github.com/mlr-org/mlr3extralearners/pull/251), which added early stopping to the survival xgboost learner, I have two questions regarding the early stopping implementation:

https://github.com/mlr-org/mlr3learners/blob/cb8162a4c85ef7183b6141d1519ba27c7cba5428/R/LearnerClassifXgboost.R#L232

What happens here if the user provides a watchlist? Should we even allow to specify a watchlist as we properly support early stopping now? Stuff can go wrong, e.g. in graphlearners etc..

https://github.com/mlr-org/mlr3learners/blob/cb8162a4c85ef7183b6141d1519ba27c7cba5428/R/LearnerClassifXgboost.R#L86

What happens if we set the early stopping set to "train". Looking at the code (https://github.com/mlr-org/mlr3learners/blob/cb8162a4c85ef7183b6141d1519ba27c7cba5428/R/LearnerClassifXgboost.R#L235) it does not seem like it is covered. In the extralearners PR I suggested to disable the "train" option as nobody would use it anyway.

In summary my suggestions are:

(1) Disable "train" as allowed factor for early_stopping_set (2) disable the watchlist hyperparameter

What do you think @be-marc ?

sebffischer avatar Nov 28 '22 10:11 sebffischer