Evaluation error: Unable to access object (object is from previous session and is now invalid)

[david-cros]

Dear all,

I have a script that worked well in RStudio on windows and RStudio server on a linux HPC. It tunes a multi-layer perceptron with ParBayesianOptimization. I tried to make it work in parallel, this way:

library(doParallel) cl <- makeCluster(2) registerDoParallel(cl) clusterExport(cl,c('OUT','X_trII', 'y_trII',"Xtr_val_data","y_trn_val_data",'my_pearson_corr','my_negative_pearson_corr','accuracy_metric')) clusterEvalQ(cl,expr= { library(keras) ; library(reticulate)})

optObj <- bayesOpt( FUN = MLP_model_to_tune, bounds = bounds, initPoints = 7, iters.n = 4, iters.k = 2, kappa=2.56,eps=0, parallel=TRUE, errorHandling = "continue", plotProgress = FALSE, verbose = 1)

stopCluster(cl) registerDoSEQ()

but it fails:

Running initial scoring function 7 times in 2 thread(s)... 11.89 seconds dropout1 dropout2 units_Fl percent_dec nDenseLayers learning_rate 1: 0.34923411 0.29540303 278.37521 0.20122780 2.442593 0.073281385 2: 0.48399678 0.54744284 265.57111 0.48144830 13.217194 0.024877648 3: 0.06770692 0.07871296 183.41782 0.01590757 11.387769 0.001059684 4: 0.53342713 0.18634436 195.52110 0.21755177 15.370336 0.061193132 5: 0.17493806 0.44141748 36.67472 0.09760904 4.983159 0.034517230 6: 0.11868415 0.39541708 67.34149 0.41009950 8.383524 0.099192594 7: 0.28663647 0.08719229 139.72736 0.35239330 6.735646 0.056862360 errorMessage 1: Evaluation error: Unable to access object (object is from previous session and is now invalid). 2: Evaluation error: Unable to access object (object is from previous session and is now invalid). 3: Evaluation error: Unable to access object (object is from previous session and is now invalid). 4: Evaluation error: Unable to access object (object is from previous session and is now invalid). 5: Evaluation error: Unable to access object (object is from previous session and is now invalid). 6: Evaluation error: Unable to access object (object is from previous session and is now invalid). 7: Evaluation error: Unable to access object (object is from previous session and is now invalid).

Error in bayesOpt(FUN = MLP_model_to_tune, bounds = bounds, initPoints = 7, : Errors encountered in initialization are listed above.

Do you have an idea about what can cause this error ?

Thanks a lot,

David