mlrMBO icon indicating copy to clipboard operation
mlrMBO copied to clipboard

"Human in the Loop MBO" fails with multiobjective methods "parego" and "nsga2"

Open mb706 opened this issue 4 years ago • 2 comments

parego fails:

ps = makeParamSet(
  makeNumericParam("q", lower = -1, upper = 2),
  makeIntegerParam("v", lower = -2, upper = 3)
)
des = generateDesign(n = 7, par.set = ps)
des$y_1 = c(1.20, 0.97, 0.91, 3.15, 0.58, 1.12, 0.50)
des$y_2 = c(1.20, 0.97, 0.91, 3.15, 0.58, 1.12, 0.50)
ctrl = makeMBOControl(n.objectives = 2)
ctrl = setMBOControlInfill(ctrl, crit = makeMBOInfillCritEI())
ctrl = setMBOControlMultiObj(ctrl, method = "parego")
opt.state = initSMBO(par.set = ps, design = des, control = ctrl, minimize = c(TRUE, TRUE), noisy = FALSE)
proposePoints(opt.state)
x = data.frame(q = 1.7, v = 1)
updateSMBO(opt.state, x = x, y = c(2.19, 2.19))

gives error

Error in control$infill.crit$fun(points = x, models = getOptStateModels(opt.state)[[1]],  : 
  Assertion on 'control$y.name' failed: Must have length 1.

similarly, mspot fails:

ps = makeParamSet(
  makeNumericParam("q", lower = -1, upper = 2),
  makeIntegerParam("v", lower = -2, upper = 3)
)
des = generateDesign(n = 7, par.set = ps)
des$y_1 = c(1.20, 0.97, 0.91, 3.15, 0.58, 1.12, 0.50)
des$y_2 = c(1.20, 0.97, 0.91, 3.15, 0.58, 1.12, 0.50)
ctrl = makeMBOControl(n.objectives = 2)
ctrl = setMBOControlInfill(ctrl, crit = makeMBOInfillCritEI(), opt = "nsga2")
ctrl = setMBOControlMultiObj(ctrl, method = "mspot")
opt.state = initSMBO(par.set = ps, design = des, control = ctrl, minimize = c(TRUE, TRUE), noisy = FALSE)
proposePoints(opt.state)
x = data.frame(q = 1.7, v = 1)
updateSMBO(opt.state, x = x, y = c(2.19, 2.19))

with error

Error in control$infill.crit$fun(points = x, models = getOptStateModels(opt.state)[[1]],  : 
  Assertion on 'control$y.name' failed: Must have length 1.

mb706 avatar Dec 19 '19 12:12 mb706

This error occurs because updateSMBO calculates the crit.vals (acquisition function values) for the given x values to write them in the opt.path. This is not essential, but a nice to have.

For the methods you mentioned (and probably more) it is not trivial to calculate them afterwards. I would suggest that we just not calculate the crit.vals in those cases. Using a tryCatch() or ifelse?

jakob-r avatar Jan 09 '20 11:01 jakob-r

Why does it not crash in mbo()?

mb706 avatar Sep 06 '20 22:09 mb706