airrflow
airrflow copied to clipboard
nf-core
Error executing process > 'NFCORE_AIRRFLOW:AIRRFLOW:CLONAL_ANALYSIS:FIND_CLONAL_THRESHOLD (all_reps)'
Description of the bug
Hi,
I am running the airrflow pipeline on bulk BCR with UMI sequencing (Tarakra bio), the overall command is below. I have tested the pipeline on just 2 samples initially and it ran without any issue. However it is failing at the clonal analysis stage; specifically at the CLONAL_ANALYSIS:FIND_CLONAL_THRESHOLD task.
The output of the .nextflow.log log file shows:
Feb-12 11:06:54.189 [TaskFinalizer-1] DEBUG nextflow.processor.TaskProcessor - Handling unexpected condition for
task: name=NFCORE_AIRRFLOW:AIRRFLOW:CLONAL_ANALYSIS:FIND_CLONAL_THRESHOLD (all_reps); work-dir=/rds/project/rds-HvBBlpwx4dc/work/53/c5a9950a380e5f0ba60c4ea9dddbf8
error [nextflow.exception.ProcessFailedException]: Process `NFCORE_AIRRFLOW:AIRRFLOW:CLONAL_ANALYSIS:FIND_CLONAL_THRESHOLD (all_reps)` terminated with an error exit status (1)
Feb-12 11:06:54.830 [TaskFinalizer-1] ERROR nextflow.processor.TaskProcessor - Error executing process > 'NFCORE_AIRRFLOW:AIRRFLOW:CLONAL_ANALYSIS:FIND_CLONAL_THRESHOLD (all_reps)'
Caused by:
Process `NFCORE_AIRRFLOW:AIRRFLOW:CLONAL_ANALYSIS:FIND_CLONAL_THRESHOLD (all_reps)` terminated with an error exit status (1)
Command executed:
Rscript -e "enchantr::enchantr_report('find_threshold', \
report_params=list('input'='find_threshold_samplesheet.txt',\
'cloneby'='subject_id',\
'crossby'='subject_id',\
'singlecell'='single_cell',\
'outdir'=getwd(),\
'nproc'=16,\
'outname'='all_reps',\
'log'='all_reps_threshold_command_log',\
'logo'='nf-core-airrflow_logo_reports.png' ,'findthreshold_cutoff'='user','findthreshold_edge'=0.9,'findthreshold_method'='gmm','findthreshold_model'='gamma-norm','findthreshold_spc'=0.995))"
cp -r enchantr all_reps_dist_report && rm -rf enchantr
echo "NFCORE_AIRRFLOW:AIRRFLOW:CLONAL_ANALYSIS:FIND_CLONAL_THRESHOLD": > versions.yml
Rscript -e "cat(paste0(' enchantr: ',packageVersion('enchantr'),'
'))" >> versions.yml
Command exit status:
1
Command output:
1/36
2/36 [global-options]
3/36
4/36 [input-parameters]
5/36 [unnamed-chunk-1]
6/36
7/36 [unnamed-chunk-2]
8/36
9/36 [unnamed-chunk-3]
10/36
11/36 [unnamed-chunk-4]
12/36 [unnamed-chunk-5]
13/36
14/36 [sum_heavy_0]
15/36
16/36 [distToNearest_bulk]
Command error:
INFO: Environment variable SINGULARITYENV_TMPDIR is set, but APPTAINERENV_TMPDIR is preferred
INFO: Environment variable SINGULARITYENV_NXF_TASK_WORKDIR is set, but APPTAINERENV_NXF_TASK_WORKDIR is preferred
INFO: Environment variable SINGULARITYENV_NXF_DEBUG is set, but APPTAINERENV_NXF_DEBUG is preferred
processing file: _main.Rmd
Error in `unserialize()`:
! error reading from connection
Backtrace:
1. bookdown::render_book(...)
2. bookdown:::render_cur_session(...)
3. rmarkdown::render(main, output_format, ..., clean = clean, envir = envir)
4. knitr::knit(knit_input, knit_output, envir = envir, quiet = quiet)
5. knitr:::process_file(text, output)
...
16. base::withRestarts(...)
17. base (local) withRestartList(expr, restarts)
18. base (local) withOneRestart(withRestartList(expr, restarts[-nr]), restarts[[nr]])
19. base (local) docall(restart$handler, restartArgs)
21. evaluate (local) fun(base::quote(`<smplErrr>`))
Warning messages:
1: replacing previous import 'data.table::first' by 'dplyr::first' when loading 'enchantr'
2: replacing previous import 'data.table::last' by 'dplyr::last' when loading 'enchantr'
3: replacing previous import 'data.table::between' by 'dplyr::between' when loading 'enchantr'
4: In normalizePath(file.path(report_params[["outdir"]], "assets", :
path[1]="assets/logo.png": No such file or directory
5: In file.copy(report_params[["logo"]], target_logo, recursive = T, :
'recursive' will be ignored as 'to' is not a single existing directory
6: In file.create(to[okay]) :
cannot create file 'assets/logo.png', reason 'No such file or directory'
Quitting from lines 221-243 [distToNearest_bulk] (_main.Rmd)
Execution halted
Work dir:
/rds/project/rds-HvBBlpwx4dc/work/53/c5a9950a380e5f0ba60c4ea9dddbf8
Container:
/home/rds/hpc-work/dependencies/Singularity/docker.io-immcantation-airrflow-4.2.0.img
Tip: when you have fixed the problem you can continue the execution adding the option `-resume` to the run command line
Feb-12 11:06:54.921 [TaskFinalizer-1] INFO nextflow.Session - Execution cancelled -- Finishing pending tasks before exit
Feb-12 11:06:54.939 [Actor Thread 19] ERROR nextflow.extension.OperatorImpl - @unknown
nextflow.exception.WorkflowScriptErrorException: Automatic clone_threshold is 'NA'. Consider setting --clonal_threshold manually.
at org.codehaus.groovy.vmplugin.v8.IndyInterface.fromCache(IndyInterface.java:321)
at nextflow.Nextflow.error(Nextflow.groovy:226)
at org.codehaus.groovy.vmplugin.v8.IndyInterface.fromCache(IndyInterface.java:321)
at Script_d520ca871aff1583$_runScript_closure1$_closure2$_closure8.doCall(Script_d520ca871aff1583:41)
at Script_d520ca871aff1583$_runScript_closure1$_closure2$_closure8.doCall(Script_d520ca871aff1583)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:343)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:328)
at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:279)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1007)
at org.codehaus.groovy.vmplugin.v8.IndyInterface.fromCache(IndyInterface.java:321)
at nextflow.extension.OperatorImpl$_ifEmpty_closure30.doCall(OperatorImpl.groovy:1043)
at jdk.internal.reflect.GeneratedMethodAccessor297.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:343)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:328)
at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:279)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1007)
at org.codehaus.groovy.vmplugin.v8.IndyInterface.fromCache(IndyInterface.java:321)
at nextflow.extension.DataflowHelper$3.afterStop(DataflowHelper.groovy:257)
at groovyx.gpars.dataflow.operator.DataflowProcessor.fireAfterStop(DataflowProcessor.java:324)
at groovyx.gpars.dataflow.operator.DataflowProcessorActor.afterStop(DataflowProcessorActor.java:59)
at jdk.internal.reflect.GeneratedMethodAccessor119.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:343)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:328)
at groovy.lang.MetaClassImpl.doInvokeMethod(MetaClassImpl.java:1333)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1088)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1007)
at org.codehaus.groovy.runtime.InvokerHelper.invokePojoMethod(InvokerHelper.java:633)
at org.codehaus.groovy.runtime.InvokerHelper.invokeMethod(InvokerHelper.java:624)
at groovyx.gpars.actor.Actor.callDynamic(Actor.java:369)
at groovyx.gpars.actor.Actor.handleTermination(Actor.java:320)
at groovyx.gpars.actor.AbstractLoopingActor.terminate(AbstractLoopingActor.java:204)
at groovyx.gpars.dataflow.operator.DataflowProcessor.terminate(DataflowProcessor.java:147)
at groovyx.gpars.dataflow.operator.DataflowProcessorActor.checkPoison(DataflowProcessorActor.java:115)
at groovyx.gpars.dataflow.operator.DataflowOperatorActor.onMessage(DataflowOperatorActor.java:83)
at groovyx.gpars.actor.impl.SDAClosure$1.call(SDAClosure.java:43)
at groovyx.gpars.actor.AbstractLoopingActor.runEnhancedWithoutRepliesOnMessages(AbstractLoopingActor.java:293)
at groovyx.gpars.actor.AbstractLoopingActor.access$400(AbstractLoopingActor.java:30)
at groovyx.gpars.actor.AbstractLoopingActor$1.handleMessage(AbstractLoopingActor.java:93)
at groovyx.gpars.util.AsyncMessagingCore.run(AsyncMessagingCore.java:132)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:840)
Feb-12 11:06:54.964 [Actor Thread 19] DEBUG nextflow.Session - Session aborted -- Cause: Automatic clone_threshold is 'NA'. Consider setting --clonal_threshold manually.
Command used and terminal output
nextflow run nf-core/airrflow -bg -r 4.2.0 \
-profile clontech_umi_bcr,cambridge \
--partition "Partition" \
--project "Account" \
--input ${Basedir}/data/nf_core_airr_sample_table.tsv \
-resume --outdir ${Outdir} \
--multiqc_title Nf-AIRR Bulk BCR Run \
--email [email protected] \
-with-report ${Outdir}/report.html &> ${Outdir}/nextflow_command.log
Relevant files
No response
System information
No response
Hi @obrien-james,
thanks for opening this issue. We are investigating why in some cases the find clonal threshold process fails. In the meanwhile, you can provide a custom clonal threshold with the parameter e.g. --clonal_threshold 0.1. We've seen that for human BCR data, a threshold of 0.1 works well in most cases.
