ERROR ~ Error executing process > 'graph_align_reads (41)'

[yykaya]

Hi,

I'm attempting to run GraffiTE in GT-sv-GA mode, but I encounter an error during the execution of the graph_align_reads (41) process, as shown in the message below. (However, when I run this step manually using GraphAligner, it completes successfully but the job gets killed.) Do you have any idea what might be causing this issue?

Tip: you can replicate the issue by changing to the process work dir and entering the command bash .command.run

-- Check '.nextflow.log' file for details

executor > local (507) [21/3da086] process > map_asm (1) [100%] 115 of 115 ✔ [cb/7c0a52] process > svim_asm (115) [100%] 115 of 115 ✔ [13/513669] process > survivor_merge [100%] 1 of 1 ✔ [b8/15025a] process > repeatmask_VCF (1) [100%] 1 of 1 ✔ [a8/d2e2df] process > tsd_prep (1) [100%] 1 of 1 ✔ [94/e3d127] process > tsd_search (148) [100%] 157 of 157 ✔ [60/be72ec] process > tsd_report (1) [100%] 1 of 1 ✔ [77/79f066] process > make_graph (1) [100%] 1 of 1 ✔ [1a/75fbbf] process > graph_align_reads (49) [100%] 115 of 115, failed: 89 ✘ [- ] process > vg_call - [- ] process > merge_VCFs - ERROR ~ Error executing process > 'graph_align_reads (41)'

Caused by: Process graph_align_reads (41) terminated with an error exit status (134)

Command executed:

GraphAligner -t 1 -x vg -g index/index.vg -f X.fastq.gz -a X.gam

vg pack -x index/index.vg -g X.gam -o X.pack -Q 0

Command exit status: 134

Command output: GraphAligner bioconda 1.0.13- Load graph from index/index.vg Build minimizer seeder from the graph Minimizer seeds, length 15, window size 20, density 10 Seed cluster size 1 Alignment bandwidth 10 Clip alignment ends with identity < 66% X-drop DP score cutoff 14705 write alignments to X.gam Align

Command error: INFO: Environment variable SINGULARITYENV_TMPDIR is set, but APPTAINERENV_TMPDIR is preferred INFO: fuse2fs not found, will not be able to mount EXT3 filesystems GraphAligner bioconda 1.0.13- GraphAligner bioconda 1.0.13- Load graph from index/index.vg Build minimizer seeder from the graph Minimizer seeds, length 15, window size 20, density 10 Seed cluster size 1 Alignment bandwidth 10 Clip alignment ends with identity < 66% X-drop DP score cutoff 14705 write alignments to X.gam Align Signal 11. Read: 2a9030c3-0fd1-4219-8496-ad2d7f1fb33b runid=b8d4812b34a5a9fdf693319045b7ee7e26775c97 read=30068 ch=2021 start_time=2019-11-30T05:26:38Z flow_cell_id=PAD94007 protocol_group_id=191128-CLIMARES sample_id=MULTIPLEX6-2-M40-S 00-L14 barcode=barcode07. Seed: 34132+,3021,15,752 .command.sh: line 2: 16 Aborted GraphAligner -t 1 -x vg -g index/index.vg -f X.fastq.gz -a X.gam

This is from .nextflow log file:

Jul-12 03:21:25.913 [Task monitor] DEBUG nextflow.processor.TaskProcessor - Handling unexpected condition for task: name=graph_align_reads (35); work-dir=/raven/ptmp/ykaya/Pangenome_project/SV/TE_SV/work/58/afee1f76707eac941eba261f63cc7d error [nextflow.exception.ProcessFailedException]: Process graph_align_reads (35) failed Jul-12 03:21:25.914 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 426; name: graph_align_reads (34); status: COMPLETED; exit: -; error: nextflow.exception.ProcessE xception: Process exceeded running time limit (12h); workDir: /raven/ptmp/ykaya/Pangenome_project/SV/TE_SV/work/39/b2a0764669a366a56a5e598a98e656] Jul-12 03:21:25.914 [Task monitor] DEBUG nextflow.processor.TaskProcessor - Handling unexpected condition for task: name=graph_align_reads (34); work-dir=/raven/ptmp/ykaya/Pangenome_project/SV/TE_SV/work/39/b2a0764669a366a56a5e598a98e656 error [nextflow.exception.ProcessFailedException]: Process graph_align_reads (34) failed Jul-12 03:21:25.915 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 440; name: graph_align_reads (48); status: COMPLETED; exit: -; error: nextflow.exception.ProcessE xception: Process exceeded running time limit (12h); workDir: /raven/ptmp/ykaya/Pangenome_project/SV/TE_SV/work/dd/a38cb1d0b16fac105668ec2bf9234a] Jul-12 03:21:25.915 [Task monitor] DEBUG nextflow.processor.TaskProcessor - Handling unexpected condition for task: name=graph_align_reads (48); work-dir=/raven/ptmp/ykaya/Pangenome_project/SV/TE_SV/work/dd/a38cb1d0b16fac105668ec2bf9234a error [nextflow.exception.ProcessFailedException]: Process graph_align_reads (48) failed Jul-12 03:21:25.915 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 396; name: graph_align_reads (4); status: COMPLETED; exit: -; error: nextflow.exception.ProcessException: Process exceeded running time limit (12h); workDir: /raven/ptmp/ykaya/Pangenome_project/SV/TE_SV/work/97/785f099c658f091c3b16c7b0abe50e] Jul-12 03:21:25.915 [Task monitor] DEBUG nextflow.processor.TaskProcessor - Handling unexpected condition for task: name=graph_align_reads (4); work-dir=/raven/ptmp/ykaya/Pangenome_project/SV/TE_SV/work/97/785f099c658f091c3b16c7b0abe50e error [nextflow.exception.ProcessFailedException]: Process graph_align_reads (4) failed Jul-12 03:21:25.916 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 434; name: graph_align_reads (42); status: COMPLETED; exit: -; error: nextflow.exception.ProcessException: Process exceeded running time limit (12h); workDir: /raven/ptmp/ykaya/Pangenome_project/SV/TE_SV/work/24/4cc79fecfa1f92029986601a22bd88] Jul-12 03:21:25.916 [Task monitor] DEBUG nextflow.processor.TaskProcessor - Handling unexpected condition for task: name=graph_align_reads (42); work-dir=/raven/ptmp/ykaya/Pangenome_project/SV/TE_SV/work/24/4cc79fecfa1f92029986601a22bd88 error [nextflow.exception.ProcessFailedException]: Process graph_align_reads (42) failed Jul-12 03:21:25.929 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 483; name: graph_align_reads (91); status: COMPLETED; exit: -; error: nextflow.exception.ProcessException: Process exceeded running time limit (12h); workDir: /raven/ptmp/ykaya/Pangenome_project/SV/TE_SV/work/fd/a12ae6a90d2858d7514f4992c60e24] Jul-12 03:21:25.929 [Task monitor] DEBUG nextflow.processor.TaskProcessor - Handling unexpected condition for task: name=graph_align_reads (91); work-dir=/raven/ptmp/ykaya/Pangenome_project/SV/TE_SV/work/fd/a12ae6a90d2858d7514f4992c60e24 error [nextflow.exception.ProcessFailedException]: Process graph_align_reads (91) failed Jul-12 03:21:25.953 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 405; name: graph_align_reads (13); status: COMPLETED; exit: -; error: nextflow.exception.ProcessException: Process exceeded running time limit (12h); workDir: /raven/ptmp/ykaya/Pangenome_project/SV/TE_SV/work/58/5a8dbb35508e8365cbef0793e7f1f1] Jul-12 03:21:25.954 [Task monitor] DEBUG nextflow.processor.TaskProcessor - Handling unexpected condition for task: name=graph_align_reads (13); work-dir=/raven/ptmp/ykaya/Pangenome_project/SV/TE_SV/work/58/5a8dbb35508e8365cbef0793e7f1f1 error [nextflow.exception.ProcessFailedException]: Process graph_align_reads (13) failed Jul-12 03:21:25.954 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 462; name: graph_align_reads (70); status: COMPLETED; exit: -; error: nextflow.exception.ProcessException: Process exceeded running time limit (12h); workDir: /raven/ptmp/ykaya/Pangenome_project/SV/TE_SV/work/58/6143156afdcd9ead22067413ce58d3] Jul-12 03:21:25.954 [Task monitor] DEBUG nextflow.processor.TaskProcessor - Handling unexpected condition for task: name=graph_align_reads (70); work-dir=/raven/ptmp/ykaya/Pangenome_project/SV/TE_SV/work/58/6143156afdcd9ead22067413ce58d3 error [nextflow.exception.ProcessFailedException]: Process graph_align_reads (70) failed Jul-12 03:21:25.955 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 484; name: graph_align_reads (92); status: COMPLETED; exit: -; error: nextflow.exception.ProcessException: Process exceeded running time limit (12h); workDir: /raven/ptmp/ykaya/Pangenome_project/SV/TE_SV/work/50/5f76f2b85046ea4410442f61413fb8] Jul-12 03:21:25.955 [Task monitor] DEBUG nextflow.processor.TaskProcessor - Handling unexpected condition for task: name=graph_align_reads (92); work-dir=/raven/ptmp/ykaya/Pangenome_project/SV/TE_SV/work/50/5f76f2b85046ea4410442f61413fb8 error [nextflow.exception.ProcessFailedException]: Process graph_align_reads (92) failed Jul-12 03:21:25.955 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 441; name: graph_align_reads (49); status: COMPLETED; exit: -; error: nextflow.exception.ProcessException: Process exceeded running time limit (12h); workDir: /raven/ptmp/ykaya/Pangenome_project/SV/TE_SV/work/1a/75fbbffbd47b5475851ec1121ee26c] Jul-12 03:21:25.956 [Task monitor] DEBUG nextflow.processor.TaskProcessor - Handling unexpected condition for task: name=graph_align_reads (49); work-dir=/raven/ptmp/ykaya/Pangenome_project/SV/TE_SV/work/1a/75fbbffbd47b5475851ec1121ee26c error [nextflow.exception.ProcessFailedException]: Process graph_align_reads (49) failed Jul-12 03:21:26.093 [main] DEBUG nextflow.Session - Session await > all barriers passed Jul-12 03:21:26.103 [main] DEBUG nextflow.util.ThreadPoolManager - Thread pool 'PublishDir' shutdown completed (hard=false) Jul-12 03:21:26.210 [main] DEBUG nextflow.trace.WorkflowStatsObserver - Workflow completed > WorkflowStats[succeededCount=418; failedCount=89; ignoredCount=0; cachedCount=0; pendingCount=0; submittedCount=0; runningCount=0; retriesCount=0; abortedCount=0; succeedDuration=9d 21h 44m 9s; failedDuration=43d 16h 36m 38s; cachedDuration=0ms;loadCpus=0; loadMemory=0; peakRunning=115; peakCpus=115; peakMemory=480 GB; ] Jul-12 03:21:26.517 [main] DEBUG nextflow.cache.CacheDB - Closing CacheDB done Jul-12 03:21:26.571 [main] DEBUG nextflow.util.ThreadPoolManager - Thread pool 'FileTransfer' shutdown completed (hard=false) Jul-12 03:21:26.571 [main] DEBUG nextflow.script.ScriptRunner - > Execution complete -- Goodbye

The command I tried:

/u/ykaya/nextflow run https://github.com/cgroza/GraffiTE --genotype true \
   --assemblies assemblies.csv \
   --TE_library TE.nonredun.fa \
   --reference /ptmp/X.fasta \
   --graph_method graphaligner \
   --reads longreads.csv