Error in rbindlist(scoreSummary) : Column 2 of item 1 is length 3 inconsistent with column 1 which is length 100. Only length-1 columns are recycled.

[marjohnsen]

I get this error right after running initial score function when initPoints is greater than a 100.

Error in rbindlist(scoreSummary) : Column 2 of item 1 is length 3 inconsistent with column 1 which is length 100. Only length-1 columns are recycled.

Simple reproducible example:

func <- function(x) list(Score = -x^2 + rnorm(n = 1, mean = 0, sd = abs(.1*x)))

bound <- list(x = c(1,10))

tune <- bayesOpt(FUN = func, bounds = bound, initPoints = 200, iters.n = 10, verbose = 2, plotProgress = T, parallel = F)

Running initial scoring function 200 times in 1 thread(s)... 28.268 seconds Error in rbindlist(scoreSummary) : Column 2 of item 1 is length 3 inconsistent with column 1 which is length 100. Only length-1 columns are recycled.

Rsession:

R version 4.1.2 (2021-11-01) Platform: x86_64-pc-linux-gnu (64-bit) Running under: Ubuntu 20.04.4 LTS

Matrix products: default BLAS: /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.9.0 LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.9.0

locale: [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C LC_TIME=nb_NO.UTF-8 LC_COLLATE=en_US.UTF-8 LC_MONETARY=nb_NO.UTF-8
[6] LC_MESSAGES=en_US.UTF-8 LC_PAPER=nb_NO.UTF-8 LC_NAME=C LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=nb_NO.UTF-8 LC_IDENTIFICATION=C

attached base packages: [1] stats graphics grDevices utils datasets methods base

other attached packages: [1] ParBayesianOptimization_1.2.5

loaded via a namespace (and not attached): [1] Rcpp_1.0.8 lubridate_1.8.0 lattice_0.20-45 tidyr_1.2.0 listenv_0.8.0 class_7.3-20 assertthat_0.2.1
[8] digest_0.6.29 ipred_0.9-12 foreach_1.5.2 utf8_1.2.2 parallelly_1.30.0 R6_2.5.1 cellranger_1.1.0
[15] plyr_1.8.6 backports_1.4.1 stats4_4.1.2 ggplot2_3.3.5 pillar_1.7.0 rlang_1.0.1 caret_6.0-90
[22] readxl_1.3.1 data.table_1.14.2 car_3.0-12 rpart_4.1.16 Matrix_1.4-0 splines_4.1.2 gower_1.0.0
[29] stringr_1.4.0 munsell_0.5.0 broom_0.7.12 compiler_4.1.2 pkgconfig_2.0.3 DiceKriging_1.6.0 globals_0.14.0
[36] nnet_7.3-17 tidyselect_1.1.2 tibble_3.1.6 prodlim_2019.11.13 codetools_0.2-18 fansi_1.0.2 future_1.23.0
[43] crayon_1.5.0 dplyr_1.0.8 withr_2.4.3 ggpubr_0.4.0 MASS_7.3-55 recipes_0.1.17 ModelMetrics_1.2.2.2 [50] grid_4.1.2 nlme_3.1-155 jsonlite_1.8.0 gtable_0.3.0 lifecycle_1.0.1 DBI_1.1.2 magrittr_2.0.2
[57] pROC_1.18.0 scales_1.1.1 carData_3.0-5 future.apply_1.8.1 cli_3.2.0 stringi_1.7.6 dbscan_1.1-10
[64] ggsignif_0.6.3 reshape2_1.4.4 timeDate_3043.102 lhs_1.1.4 ellipsis_0.3.2 generics_0.1.2 vctrs_0.3.8
[71] xgboost_1.5.0.2 lava_1.6.10 iterators_1.0.14 tools_4.1.2 glue_1.6.2 purrr_0.3.4 abind_1.4-5
[78] parallel_4.1.2 survival_3.2-13 colorspace_2.0-3 rstatix_0.7.0

This happens for any function with initPoints greater than a 100. Works perfectly at 100, but that is not a sufficient number of initial points.

[dschmidt-hgu]

I ran into the same error and found that this issue is related to the foreach function, more precisely the .multicombine=TRUE option that is linked to .maxcombine = if (.multicombine) 100 else 2 Where an upper limit of 100 is hardcoded.

From the help:

foreach(
  ...,
  .combine,
  .init,
  .final = NULL,
  .inorder = TRUE,
  .multicombine = FALSE,
  .maxcombine = if (.multicombine) 100 else 2,
  .errorhandling = c("stop", "remove", "pass"),
  .packages = NULL,
  .export = NULL,
  .noexport = NULL,
  .verbose = FALSE
)

I could temporarily fix this issue by including , .maxcombine= nrow(initGrid) in bayesOpt.R:

  # Run initialization
  if (verbose > 0) cat("\nRunning initial scoring function",nrow(initGrid),"times in",Workers,"thread(s)...")
  sink(file = sinkFile)
  tm <- system.time(
    scoreSummary <- foreach(
        iter = 1:nrow(initGrid)
      , .options.multicore = list(preschedule=FALSE)
      , .combine = list
      , .multicombine = TRUE
      , .maxcombine= nrow(initGrid)      #newline
      , .inorder = FALSE
      , .errorhandling = 'pass'
      #, .packages ='data.table'
      , .verbose = FALSE
    ) %op% {