xcms Error in result[[njob]] <- value: attempt to select less than one element in OneIndex

Error in result[[njob]] <- value: attempt to select less than one element in OneIndex

Open AharoniLab opened this issue 1 year ago • 12 comments

Hi,

I am getting the same cryptic error mentioned a couple of years ago

With the following added details:

reverting the older XCMS code on the same data does not produce the error
re-running using serial processing does not trigger this error
parallel processing was run with just 2 cores on a small experiment (<20 samples) on a high-performance cluster - thus I/O is probably not the cause for all processes being killed by the system
Waters raw data format was converted to 'mzXML' with split MS channels (i.e. ms1 and ms2 are exported to different files)
using XCMS 3.8.2 with R version 3.6.3 on Ubuntu 20.04 LTS

Any update or feedback would be welcome.

Thanks!

Nov 02 '22 08:11 AharoniLab

Hi, could you link any other issue you refer to ? Could you give a code snippet that triggered the issue ? Can you run traceback() after the error happened ? Yours, Steffen

Nov 02 '22 09:11 sneumann

Referring to:

https://github.com/sneumann/xcms/issues/437

The following command already triggers this error. 'rd_pools' in this case is an "OnDiskMSnExp" object linked to four experimental files:

xd_pools = findChromPeaks (rd_pools,cwp,BPPARAM = bpparam$MulticoreParam) | | 0%Error in result[[njob]] <- value : attempt to select less than one element in OneIndex In addition: Warning message: In parallel::mccollect(wait = FALSE, timeout = 1) : 1 parallel job did not deliver a result

traceback() 7: bploop.lapply(cls, X, lapply, ARGFUN, BPPARAM) 6: bploop(cls, X, lapply, ARGFUN, BPPARAM) 5: bplapply(object_mslevel, FUN = findChromPeaks_OnDiskMSnExp, method = "centWave", param = param, BPPARAM = BPPARAM) 4: bplapply(object_mslevel, FUN = findChromPeaks_OnDiskMSnExp, method = "centWave", param = param, BPPARAM = BPPARAM) 3: .local(object, param, ...) 2: findChromPeaks(rd_pools, cwp, BPPARAM = bpparam$MulticoreParam) 1: findChromPeaks(rd_pools, cwp, BPPARAM = bpparam$MulticoreParam)`

Thanks, Nir.

Nov 02 '22 14:11 AharoniLab

Thanks Nir for the additional information. Unfortunately I do not have any direct suggestion. There is https://bioconductor.org/packages/release/bioc/vignettes/BiocParallel/inst/doc/Errors_Logs_And_Debugging.pdf and things to try could be adding sink(NULL, type = "message") (see 2.1 Messages and warnings) into https://github.com/sneumann/xcms/blob/af1e8b9e79b6039a34951e9e79329d99ce778397/R/methods-OnDiskMSnExp.R#L83 and/or getting the traceback via the instructions in "2.3 Identify failures with bpok()" using the equivalent of result <- bptry(bplapply(list(1, "2", 3), sqrt, BPPARAM=param)) and tail(attr(result[[which(!bpok(result))]], "traceback. If any of that gives (more) informative feedback, that'd be really great. Any idea how I could provoke the error here on my machine ? Yours, Steffen

Nov 02 '22 16:11 sneumann

Hm, maybe there is a simpler way to get the debug info:

bpparam <- MulticoreParam(workers = 4,
	stop.on.error = TRUE,
	log = TRUE, threshold = "INFO", logdir = "/tmp")

but I got empty log files in /tmp , maybe because I was unable to trigger the issue. Yours, Steffen

Nov 02 '22 17:11 sneumann

Hi Nir,

since you don't get the error with SerialParam but only with MulticoreParam I suspect it might have to do with the cluster settings. On our HPC we use slurm as queueing system and usually all works well, only thing is I need to make sure that all parallel processes are run on the same node. MulticoreParam uses shared memory, thus it will not work if the processes are run on different nodes (servers/computers). Also, make sure you assign enough memory to each node/task - and obviously also each node (process) needs to have access to the raw data file (same path). Maybe you could also try to use SnowParam as an alternative?

Nov 04 '22 09:11 jorainer

Hi,

Thanks both for your replies.

running with SnowParam did not work either and resulted in the following error: Error in serialize(data, node$con) : ignoring SIGPIPE signal Calls: local ... .send -> <Anonymous> -> sendData.SOCKnode -> serialize Execution halted
logging from MulticoreParam did not result in any log files on this server, but did generate the per process logs with the run which completely successfully on the non-cluster PC.
generating logs using the suggested bptry on bpapply did not work either as findChromPeaks kept throwing errors regarding the split object type:

rd_pools_byFile = split (rd_pools,f=fromFile (rd_pools))
result <- bptry (
	bplapply (rd_pools_byFile,findChromPeaks,BPPARAM=bpparam)
)

Error in (function (classes, fdef, mtable) : unable to find an inherited method for function ‘findChromPeaks’ for signature ‘"OnDiskMSnExp", "missing"’

which is maybe caused by improper splitting or some missing preprocessing of the OnDiskMSnExp object?!

Nov 07 '22 09:11 AharoniLab

Can you please provide the output of sessionInfo() from the cluster as well the local (working) system?

I got also a bit confused by the findChromPeaks(rd_pools, cwp, BPPARAM = bpparam$MulticoreParam) call - ideally you should run it either with

register(bpstart(MulticoreParam(4)))
res <- findChromPeaks(rd_pools, cwp)

res <- findChromPeaks(rd_pools, cwp, BPPARAM = MulticoreParam(4))

The first option should be preferred as you define and initialize the parallel processing setup globally for your R session. Any xcms function supporting parallel processing will automatically use this default setup.

Regarding the error you got: the problem is that you forgot to pass the cwp parameter. That's why the error says unable to find an inherited... for signature "OnDiskMSnExp", "missing". You should rather call

rd_pools_byFile = split (rd_pools,f=fromFile (rd_pools))
result <- bptry (
	bplapply (rd_pools_byFile,findChromPeaks, param = cwp, BPPARAM=bpparam)
)

Nov 14 '22 13:11 jorainer

Having my own parallel processing problems on a linux system in a conda environment. Using an older version of R/XCMS/BioC works fine on the same computer. I am pretty ignorant when it comes to HPC, so maybe i am failing to set something properly, but the fact that it works with older R suggests that something about BioC paralell processing has changed.

proj.dir <- "20220310-CBROE-CB-1173/PH_QC_Subset/" library(xcms) #register(bpstart(MulticoreParam(workers = 1, tasks = 12))) ## tried this to no availl register(bpstart(MulticoreParam(workers = 12))) setwd(paste0("/home/pmflab/RStor/Projects/", proj.dir)) seq <- read.csv('seq_QCs.csv', header = TRUE, check.names = FALSE) # seq <- seq[1:2,] mzml <- c(paste0(as.character(seq[,1]), ".mzmL")) ## 28 files for testing

pd <- data.frame("filename" = mzml, seq[,2]) qc <- grepl("QC", seq[,2], ignore.case = TRUE) sample.groups <- as.integer(qc)

cw.param <- CentWaveParam( ppm = 25, peakwidth = c(2.2, 15),
snthresh = 10, prefilter = c(3, 10), mzCenterFun = "wMean", integrate = 1L, mzdiff = 0.015, fitgauss = TRUE, noise = 2, verboseColumns = TRUE, roiList = list(), firstBaselineCheck = TRUE, roiScales = numeric(), extendLengthMSW = TRUE )

raw_data <- readMSData(files = mzml, pdata = new("NAnnotatedDataFrame", pd), mode = "onDisk") x.data <- findChromPeaks(raw_data, param = cw.param)

Gives an error: Error in serialize(data, node$con, xdr = FALSE) : ignoring SIGPIPE signal Error in serialize(data, node$con, xdr = FALSE) : error writing to connection

this is operating in a Conda environment, R 4.2.2.
sessionInfo() R version 4.2.2 (2022-10-31) Platform: x86_64-conda-linux-gnu (64-bit) Running under: Ubuntu 20.04.4 LTS

Matrix products: default BLAS/LAPACK: /home/pmflab/miniconda3/envs/r422/lib/libopenblasp-r0.3.21.so

locale: [1] C

attached base packages: [1] stats4 stats graphics grDevices utils datasets methods [8] base

other attached packages: [1] xcms_3.20.0 MSnbase_2.24.0 ProtGenerics_1.30.0 [4] S4Vectors_0.36.0 mzR_2.32.0 Rcpp_1.0.9 [7] Biobase_2.58.0 BiocGenerics_0.44.0 BiocParallel_1.32.0

loaded via a namespace (and not attached): [1] lattice_0.20-45 assertthat_0.2.1 [3] digest_0.6.30 foreach_1.5.2 [5] utf8_1.2.2 R6_2.5.1 [7] GenomeInfoDb_1.34.1 plyr_1.8.8 [9] mzID_1.36.0 ggplot2_3.4.0 [11] pillar_1.8.1 zlibbioc_1.44.0 [13] rlang_1.0.6 Matrix_1.5-3 [15] preprocessCore_1.60.0 RCurl_1.98-1.9 [17] munsell_0.5.0 DelayedArray_0.24.0 [19] compiler_4.2.2 MsFeatures_1.6.0 [21] pkgconfig_2.0.3 pcaMethods_1.90.0 [23] tidyselect_1.2.0 SummarizedExperiment_1.28.0 [25] tibble_3.1.8 GenomeInfoDbData_1.2.9 [27] RANN_2.6.1 IRanges_2.32.0 [29] codetools_0.2-18 matrixStats_0.62.0 [31] XML_3.99-0.12 fansi_1.0.3 [33] dplyr_1.0.10 MASS_7.3-58.1 [35] bitops_1.0-7 MassSpecWavelet_1.64.0 [37] grid_4.2.2 gtable_0.3.1 [39] lifecycle_1.0.3 affy_1.76.0 [41] DBI_1.1.3 magrittr_2.0.3 [43] MsCoreUtils_1.10.0 scales_1.2.1 [45] ncdf4_1.19 cli_3.4.1 [47] impute_1.72.0 XVector_0.38.0 [49] affyio_1.68.0 doParallel_1.0.17 [51] limma_3.54.0 robustbase_0.95-0 [53] generics_0.1.3 vctrs_0.5.1 [55] RColorBrewer_1.1-3 iterators_1.0.14 [57] tools_4.2.2 glue_1.6.2 [59] DEoptimR_1.0-11 MatrixGenerics_1.10.0 [61] parallel_4.2.2 clue_0.3-62 [63] colorspace_2.0-3 cluster_2.1.4 [65] BiocManager_1.30.19 vsn_3.66.0 [67] GenomicRanges_1.50.0 MALDIquant_1.22

Nov 22 '22 17:11 cbroeckl

This conda set up worked on the same computer: sessionInfo() R version 4.1.2 (2021-11-01) Platform: x86_64-conda-linux-gnu (64-bit) Running under: Ubuntu 20.04.4 LTS

Matrix products: default BLAS/LAPACK: /home/pmflab/miniconda3/envs/xcms/lib/libopenblasp-r0.3.18.so

locale: [1] C

attached base packages: [1] stats4 stats graphics grDevices utils datasets methods [8] base

other attached packages: [1] xcms_3.16.1 MSnbase_2.20.4 ProtGenerics_1.26.0 [4] S4Vectors_0.32.3 mzR_2.28.0 Rcpp_1.0.8 [7] Biobase_2.54.0 BiocGenerics_0.40.0 BiocParallel_1.28.3

loaded via a namespace (and not attached): [1] lattice_0.20-45 assertthat_0.2.1 [3] digest_0.6.29 foreach_1.5.2 [5] utf8_1.2.2 R6_2.5.1 [7] GenomeInfoDb_1.30.1 plyr_1.8.6 [9] mzID_1.32.0 ggplot2_3.3.5 [11] pillar_1.7.0 zlibbioc_1.40.0 [13] rlang_1.0.1 Matrix_1.4-0 [15] preprocessCore_1.56.0 RCurl_1.98-1.6 [17] munsell_0.5.0 DelayedArray_0.20.0 [19] compiler_4.1.2 MsFeatures_1.2.0 [21] pkgconfig_2.0.3 pcaMethods_1.86.0 [23] tidyselect_1.1.1 SummarizedExperiment_1.24.0 [25] GenomeInfoDbData_1.2.7 tibble_3.1.6 [27] RANN_2.6.1 IRanges_2.28.0 [29] codetools_0.2-18 matrixStats_0.61.0 [31] XML_3.99-0.8 fansi_1.0.2 [33] crayon_1.4.2 dplyr_1.0.8 [35] bitops_1.0-7 MASS_7.3-55 [37] MassSpecWavelet_1.60.0 grid_4.1.2 [39] gtable_0.3.0 lifecycle_1.0.1 [41] affy_1.72.0 DBI_1.1.2 [43] magrittr_2.0.2 MsCoreUtils_1.6.0 [45] scales_1.1.1 ncdf4_1.19 [47] cli_3.1.1 impute_1.68.0 [49] XVector_0.34.0 affyio_1.64.0 [51] doParallel_1.0.17 limma_3.50.0 [53] robustbase_0.93-9 ellipsis_0.3.2 [55] generics_0.1.2 vctrs_0.3.8 [57] RColorBrewer_1.1-2 iterators_1.0.14 [59] tools_4.1.2 glue_1.6.1 [61] DEoptimR_1.0-10 purrr_0.3.4 [63] MatrixGenerics_1.6.0 parallel_4.1.2 [65] clue_0.3-60 colorspace_2.0-2 [67] cluster_2.1.2 BiocManager_1.30.16 [69] vsn_3.62.0 GenomicRanges_1.46.1 [71] MALDIquant_1.21

Nov 22 '22 20:11 cbroeckl

i was able to get this to work in Linux in new xcms/Bioc by moving to Snow:

Success: register(bpstart(SnowParam(workers = 12))) Fails: register(bpstart(MulticoreParam(workers = 12)))

sessionInfo() R version 4.2.2 (2022-10-31) Platform: x86_64-conda-linux-gnu (64-bit) Running under: Ubuntu 20.04.4 LTS

Matrix products: default BLAS/LAPACK: /home/pmflab/miniconda3/envs/r422/lib/libopenblasp-r0.3.21.so

locale: [1] C

attached base packages: [1] stats4 stats graphics grDevices utils datasets methods [8] base

other attached packages: [1] xcms_3.20.0 MSnbase_2.24.0 ProtGenerics_1.30.0 [4] S4Vectors_0.36.0 mzR_2.32.0 Rcpp_1.0.9 [7] Biobase_2.58.0 BiocGenerics_0.44.0 BiocParallel_1.32.0

Nov 23 '22 16:11 cbroeckl

Hi back,

Sorry for the pause. Following the info which Steffen sent, the sys. admin has limited this instance's use of resources to 80% and this has already made the command work as-is also when using multi cores. Ee also set the VM "overcommit_ratio" to zero - but I cannot say if this directly resolved the issue without the extra limits on the VM.

@jorainer

register(bpstart(MulticoreParam(4))) res <- findChromPeaks(rd_pools, cwp)

I initially used this setup, but I have also had occasional problems when running 'chromatogram' and 'fillChromPeaks' with the multi-core option - so I resolved to running different XCMS functions with different multi/series-core settings. Still 'bptry' on the split data might be a better way to deal with very variable system loads and possible crashes in one of the threads.

Regarding the error you got: the problem is that you forgot to pass the cwp parameter. That's why the error says unable to find an inherited... for signature "OnDiskMSnExp", "missing

I could not find any missing 'cwp' parameter - but regardless you must admit that the error message does not contribute much to debugging the problem...

Cluster VM info: `> sessionInfo() R version 3.6.3 (2020-02-29) Platform: x86_64-pc-linux-gnu (64-bit) Running under: Debian GNU/Linux bullseye/sid

Matrix products: default BLAS: /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.9.0 LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.9.0

locale: [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C [3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8 [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8 [7] LC_PAPER=en_US.UTF-8 LC_NAME=C [9] LC_ADDRESS=C LC_TELEPHONE=C [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C

attached base packages: [1] stats graphics grDevices utils datasets methods base `

Local PC info: `> sessionInfo() R version 3.6.3 (2020-02-29) Platform: x86_64-pc-linux-gnu (64-bit) Running under: Ubuntu 18.04.6 LTS

Matrix products: default BLAS: /usr/lib/x86_64-linux-gnu/openblas/libblas.so.3 LAPACK: /usr/lib/x86_64-linux-gnu/libopenblasp-r0.2.20.so

attached base packages: [1] grid stats4 parallel stats graphics grDevices utils [8] datasets methods base

other attached packages: [1] Rdisop_1.46.0 classyfireR_0.3.6 magrittr_2.0.1 [4] gridExtra_2.3 readxl_1.3.1 RColorBrewer_1.1-2 [7] XML_3.99-0.3 CAMERA_1.42.0 xcms_3.8.2 [10] MSnbase_2.12.0 ProtGenerics_1.18.0 S4Vectors_0.24.4 [13] mzR_2.20.0 Rcpp_1.0.6 BiocParallel_1.20.1 [16] Biobase_2.46.0 BiocGenerics_0.32.0

loaded via a namespace (and not attached): [1] vsn_3.54.0 splines_3.6.3 foreach_1.5.1 [4] Formula_1.2-4 BiocManager_1.30.18 affy_1.64.0 [7] latticeExtra_0.6-29 RBGL_1.62.1 cellranger_1.1.0 [10] robustbase_0.93-7 impute_1.60.0 pillar_1.4.7 [13] backports_1.2.1 lattice_0.20-45 glue_1.4.2 [16] limma_3.42.2 digest_0.6.27 checkmate_2.0.0 [19] colorspace_2.0-0 htmltools_0.5.1.1 preprocessCore_1.48.0 [22] Matrix_1.3-4 plyr_1.8.6 MALDIquant_1.19.3 [25] pkgconfig_2.0.3 zlibbioc_1.32.0 purrr_0.3.4 [28] scales_1.1.1 RANN_2.6.1 jpeg_0.1-8.1 [31] affyio_1.56.0 tibble_3.0.6 htmlTable_2.1.0 [34] generics_0.1.0 IRanges_2.20.2 ggplot2_3.3.3 [37] ellipsis_0.3.1 nnet_7.3-16 MassSpecWavelet_1.52.0 [40] survival_3.2-13 crayon_1.4.1 ncdf4_1.17 [43] doParallel_1.0.16 MASS_7.3-54 foreign_0.8-76 [46] graph_1.64.0 data.table_1.13.6 tools_3.6.3 [49] lifecycle_1.0.0 stringr_1.4.0 munsell_0.5.0 [52] cluster_2.1.1 pcaMethods_1.78.0 compiler_3.6.3 [55] mzID_1.24.0 rlang_0.4.10 iterators_1.0.13 [58] rstudioapi_0.13 htmlwidgets_1.5.3 igraph_1.2.6 [61] base64enc_0.1-3 gtable_0.3.0 codetools_0.2-18 [64] multtest_2.42.0 R6_2.5.0 knitr_1.31 [67] dplyr_1.0.4 Hmisc_4.4-2 stringi_1.5.3 [70] vctrs_0.3.6 rpart_4.1-15 png_0.1-7 [73] DEoptimR_1.0-8 tidyselect_1.1.0 xfun_0.21`

Nov 28 '22 12:11 AharoniLab

I could not find any missing 'cwp' parameter - but regardless you must admit that the error message does not contribute much to debugging the problem...

sorry, I was maybe not clear. What I meant is the following:

the code you used and that has thrown this error message (for signature "OnDiskMSnExp", "missing") was:

rd_pools_byFile = split (rd_pools,f=fromFile (rd_pools))
result <- bptry (
	bplapply (rd_pools_byFile,findChromPeaks,BPPARAM=bpparam)
)

so, in fact you are applying findChromPeaks to each element of rd_pools_byFile, but you are not passing the parameter object to the findChromPeaks function. So, in essence, in each loop you are calling findChromPeaks(rd_pools_byFile[[i]]) and findChromPeaks cries that it can not find the parameter object (param is in fact "missing", that's what the cryptic error message means).

So, you would need to pass also the cwp with parameter param to findChromPeaks:

rd_pools_byFile = split (rd_pools,f=fromFile (rd_pools))
result <- bptry (
	bplapply (rd_pools_byFile, findChromPeaks, param = cwp, BPPARAM=bpparam)
)

hope this makes sense

Nov 30 '22 11:11 jorainer

xcms xcms copied to clipboard

Error in result[[njob]] <- value: attempt to select less than one element in OneIndex

xcms
xcms copied to clipboard