paradox icon indicating copy to clipboard operation
paradox copied to clipboard

transforming 2 levels ParamUty class in nested cross-validation

Open The-Killbill opened this issue 3 years ago • 0 comments

'num_nodes' hyperparameter in Param_set of 'surv.deephit' learner is a ParamUty class with default value: 32, 32. To optimize that with two random num_nodes, I wrote the code below:

search_space <- ps(num_nodes = p_fct(list(c(32,64,128,256)), trafo = function(x) c(sample(x,1), sample(x,1))))

to check that:

generate_design_random(search_space,10)$transpose()

Which showed:

[[1]]$num_nodes
[1] 64 128

[[2]]$num_nodes
[1] 32 256

...

But when I added it to a nested cross validation model to find its optimized value , It included all factors instead of randomly selected two of them, briefly as follows (just include column of num_nodes):

# Code :
...
#defining autotuner
at <- AutoTuner$new(dh.learner, resampling, measure, terminator, tuner, search_space)

#outer cross validation
resamplin_outer <- rsmp('cv', folds = 3)

# nested resampling
nest_rsm <- resample(task.mlr, at, resampling_outer)

Which showed:

  num_nodes 
   c(32,64,128,256)
    num_nodes
   c(32,64,128,256)
   ... 

So, how is it possible to transform parameters with ParamUty class with more than one level, which can be applied to nested cross validation?

The-Killbill avatar Apr 17 '21 08:04 The-Killbill