mlr3proba icon indicating copy to clipboard operation
mlr3proba copied to clipboard

OOB with ranger: Error during tuning for TTE endpoints

Open MasterLuke84 opened this issue 3 years ago • 2 comments

Hi Raphael,

I hope this is the right repo for this issue.

When using oob_error as performance measure for TTE endpoint with the ranger engine this error occurs:

image

It works with regression/classification endpoints.

Example

library(mlr3)
library(paradox)
library(mlr3proba)
library(survival)

data("cancer")
data <- cancer %>% na.omit()

params <- ps(
  mtry =          p_int(lower = 1, upper = 3), 
  min.node.size = p_int(lower = 1, upper = 2), 
  num.trees =     p_int(lower = 50, upper = 100))

grid <- generate_design_lhs(params, n = 3)

mlr3tuning::tune(method       = "design_points", 
                 learner      = lrn("surv.ranger"), 
                 task         = as_task_surv(data, time = "time", event = "status",  id = "1111"), 
                 resampling   = mlr3::rsmp("cv", folds = 4), 
                 measures     = mlr3::msr("oob_error"), 
                 design       = grid$data, 
                 store_models = TRUE,
                 search_space = params)

Best regards

MasterLuke84 avatar May 02 '22 10:05 MasterLuke84

Thanks. It's breaking because of incompatible predict types, I might have to implement surv.oob_error to fix this.

RaphaelS1 avatar May 06 '22 08:05 RaphaelS1

At some point it has already worked. If you run the example with following package versions, no issues occur.

mlr3 0.12.0
mlr3learners 0.5.0
mlr3proba 0.4.1
mlr3tuning 0.9.0
mlr3measures 0.4.0

Maybe this might help to narrow down the issue.

Thanks in advance and enjoy the weekend!

MasterLuke84 avatar May 06 '22 12:05 MasterLuke84

@RaphaelS1 you can close this (fixed in https://github.com/mlr-org/mlr3/issues/816)

bblodfon avatar Feb 03 '23 16:02 bblodfon