Any way to add constraints in between the parameters?

[actuarial-lonewolf]

Hi,

First off, thanks for your great package! I have used it to tweak hyperparameters.

I thought about your package to solve an ensembling optimization issue.

I was wondering if it was possible to easily tweak your algorithm to incorporate constraints?

As an example, let's say I have predictions returned from 4 different models. I would like to find the optimal weights w1 to w4 that minimize the RMSE.

  library(ParBayesianOptimization)
  
  
  set.seed(123)
  model1 = rnorm(50, mean=20, sd=1)
  model2 = rnorm(50, mean=30, sd=1)
  model3 = rnorm(50, mean=40, sd=1)
  model4 = rnorm(50, mean=50, sd=1)
  actual = rnorm(50, mean=33, sd=1)
  
  
  
  scoringFunction <- function(w1, w2, w3, w4) {
    
      result     =  (  w1   * model1 +
                       w2   * model2 +
                       w3   * model3 +
                       w4   * model4)
    
    result.rmse = sqrt(mean(  (result -  actual ) ^2))
    
    
    return(list(Score = -result.rmse) )
  }
  
  
  
  
  bounds <- list( w1    = c(0, 1)
                , w2    = c(0, 1)
                , w3    = c(0, 1)
                , w4    = c(0, 1)
  
  )

 # Some code to tell bayesOpt that while it is free to roam in the bounds,
  # the sum of the parameters must always equal 1 ? 

  #constraint = (w1 + w2 + w3 + w4 == 1)
  
  
  
  t <- system.time(
    optObj <- bayesOpt( FUN = scoringFunction
                        , bounds = bounds
                        , initPoints = 10  
                        , iters.n = 30
                        , iters.k = 1 
                        , parallel = FALSE
                        #, plotProgress = TRUE
                        , otherHalting = list(timeLimit = 600) #time limit in second 
    )
  )

Running this code above and then fetching the best parameters

      > getBestPars(optObj)
      $w1
      [1] 0.05068397
      
      $w2
      [1] 0.3165851
      
      $w3
      [1] 0.1294854
      
      $w4
      [1] 0.3349892

As you can see, it is not constrained to have the parameters sum to 1, which is expected.

Anyways, I'm reaching out to see if you can see a fast approach to this problem.

Thanks again!

[AnotherSamWilson]

Hmmm this is an interesting suggestion. I know how I would solve it - just put constraints on the acquisition function. However, this is not possible as of now. I'll look into some options.

[swaheera]

@AnotherSamWilson : can you please explain how would someone add constraints to the acquisition function? thanks

[AnotherSamWilson]

It is not currently possible - right now the process searches anywhere within the parameter bounds for the acquisition function optimum. I looked into this a little bit and it would end up being quite complicated, since the type of constraints can be different from problem to problem. I'm open to suggestions on how this should be done.