JAFFA
JAFFA copied to clipboard
Published
20 hours ago •
Oshlack
Oshlack
Pipeline fails at multiple stages on simulated nanopore dataset: mismatch between length of bases and qualities for reads
Hello, I am currently testing JAFFAL and other long-read fusion finding tools on a simulated dataset ~generated using badread to simulate reads and FUSIM to simulate fusions (including tri-fusions). The command I use:
/opt/JAFFA-version-2.3/tools/bin/bpipe run -n 3 /opt/JAFFA-version-2.3/JAFFAL.groovy ../*positive_control_nanopore2020_D*.fastq.gz
The pipeline runs successfully and (relatively) smoothly on my spiked-in simulated pacbio dataset. except for this error message when entering Stage report_3_gene_fusions :
read id: fd35739b-5ecc-ca47-d0fd-05d771853c07 does not conform to 3 gene fusion structure.
(and I get this type of message for several reads in several data files). However, my main issue is when I run JAFFAL on my spiked-in simulated nanopore dataset where the pipeline fails, and I do not get a jaffa_results.csv or .fasta output file.
========================================= Pipeline Failed ==========================================
One or more parallel stages aborted. The following messages were reported:
Branch positive_control_nanopore2020_D100x_M90.fastq in stage Unknown reported message:
Command failed with exit status = 1 :
/opt/JAFFA-version-2.3/tools/bin/reformat ignorebadquality=t in=../positive_control_nanopore2020_D100x_M90.fastq.gz out=positive_control_nanopore2020_D100x_M90.fastq/positive_control_nanopore2020_D100x_M90.fastq.fasta threads=1 ;
Branch positive_control_nanopore2020_D20x_M85.fastq in stage Unknown reported message:
Command failed with exit status = 1 :
/opt/JAFFA-version-2.3/tools/bin/reformat ignorebadquality=t in=../positive_control_nanopore2020_D20x_M85.fastq.gz out=positive_control_nanopore2020_D20x_M85.fastq/positive_control_nanopore2020_D20x_M85.fastq.fasta threads=1 ;
Branch positive_control_nanopore2020_D50x_M85.fastq in stage Unknown reported message:
Command failed with exit status = 1 :
/opt/JAFFA-version-2.3/tools/bin/reformat ignorebadquality=t in=../positive_control_nanopore2020_D50x_M85.fastq.gz out=positive_control_nanopore2020_D50x_M85.fastq/positive_control_nanopore2020_D50x_M85.fastq.fasta threads=1 ;
Branch positive_control_nanopore2020_D50x_M90.fastq in stage Unknown reported message:
Command failed with exit status = 1 :
/opt/JAFFA-version-2.3/tools/bin/reformat ignorebadquality=t in=../positive_control_nanopore2020_D50x_M90.fastq.gz out=positive_control_nanopore2020_D50x_M90.fastq/positive_control_nanopore2020_D50x_M90.fastq.fasta threads=1 ;
Use 'bpipe errors' to see output from failed commands.
ryleyd@minion:/data2/ryleyd/benchmark_simulated_reads/positive_control/nanopore$ /opt/JAFFA-version-2.3/tools/bin/bpipe errors
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.codehaus.groovy.reflection.CachedClass$3$1 (file:/opt/JAFFA-version-2.3/tools/bpipe-0.9.9.2/lib/bpipe.jar) to method java.lang.Object.finalize()
WARNING: Please consider reporting this to the maintainers of org.codehaus.groovy.reflection.CachedClass$3$1
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
============================== Found 4 failed commands from run 99149 ==============================
=========================================== Command 288 ============================================
Command : /opt/JAFFA-version-2.3/tools/bin/reformat ignorebadquality=t in=../positive_control_nanopore2020_D100x_M90.fastq.gz out=positive_control_nanopore2020_D100x_M90.fastq/positive_control_nanopore2020_D100x_M90.fastq.fasta threads=1 ;
Exit Code : 1
Output :
No command output found in most recent log file
=========================================== Command 297 ============================================
Command : /opt/JAFFA-version-2.3/tools/bin/reformat ignorebadquality=t in=../positive_control_nanopore2020_D20x_M85.fastq.gz out=positive_control_nanopore2020_D20x_M85.fastq/positive_control_nanopore2020_D20x_M85.fastq.fasta threads=1 ;
Exit Code : 1
Output :
No command output found in most recent log file
=========================================== Command 352 ============================================
Command : /opt/JAFFA-version-2.3/tools/bin/reformat ignorebadquality=t in=../positive_control_nanopore2020_D50x_M85.fastq.gz out=positive_control_nanopore2020_D50x_M85.fastq/positive_control_nanopore2020_D50x_M85.fastq.fasta threads=1 ;
Exit Code : 1
Output :
No command output found in most recent log file
=========================================== Command 356 ============================================
Command : /opt/JAFFA-version-2.3/tools/bin/reformat ignorebadquality=t in=../positive_control_nanopore2020_D50x_M90.fastq.gz out=positive_control_nanopore2020_D50x_M90.fastq/positive_control_nanopore2020_D50x_M90.fastq.fasta threads=1 ;
Exit Code : 1
Output :
No command output found in most recent log file
More information on the error messages I receive while the pipeline is running and files are progressing through stages
For several (but not all) files I will run into errors like:
=== Stage minimap2_transcriptome (positive_control_nanopore2020_D10x_M90) ===
Mismatch between length of bases and qualities for read 9276039 (id=2db111e8-9f27-5a08-a383-d2af7b6d92bd ENST00000391975.5|ENSG00000115677.1 8|OTTHUMG00000133391.10|OTTHUMT00000257245.6|HDLBP-203|HDLBP|6372|protein_coding|,+strand,0-6372 length=5861 error-free_length=6383 read_ide ntity=78.63%).
# qualities=138, # bases=5861
....
at stream.Read.validate(Read.java:114)
at stream.Read.<init>(Read.java:78)
at stream.Read.<init>(Read.java:61)
at stream.FASTQ.quadToRead(FASTQ.java:872)
at stream.FASTQ.toReadList(FASTQ.java:705)
at stream.FastqReadInputStream.fillBuffer(FastqReadInputStream.java:111)
at stream.FastqReadInputStream.nextList(FastqReadInputStream.java:96)
at stream.ConcurrentGenericReadInputStream$ReadThread.readLists(ConcurrentGenericReadInputStream.java:656)
at stream.ConcurrentGenericReadInputStream$ReadThread.run(ConcurrentGenericReadInputStream.java:635)
Input: 9276000 reads 16463207847 bases
Output: 9276000 reads (100.00%) 16463207847 bases (100.00%)
Time: 251.239 seconds.
Reads Processed: 9276k 36.92k reads/sec
Bases Processed: 16463m 65.53m bases/sec
Exception in thread "main" java.lang.RuntimeException: ReformatReads terminated in an error state; the output may be corrupt.
at jgi.ReformatReads.process(ReformatReads.java:1103)
at jgi.ReformatReads.main(ReformatReads.java:43)
ERROR: Command failed with exit status = 1 :
/opt/JAFFA-version-2.3/tools/bin/reformat ignorebadquality=t in=../positive_control_nanopore2020_D100x_M90.fastq.gz out=positive_control_nan opore2020_D100x_M90.fastq/positive_control_nanopore2020_D100x_M90.fastq.fasta threads=1 ;
= Stage minimap2_transcriptome (positive_control_nanopore2020_D10x_M95) ~runs without error
== Stage filter_transcripts (positive_control_nanopore2020_D10x_M95) ==
java.lang.AssertionError:
Mismatch between length of bases and qualities for read 1934044 (id=d642ebd4-a024-95da-520b-1b29b14a4348 ENST00000361624.2|ENSG00000198804.2|-|-|MT-CO1-201|MT-CO1|1542|protein_coding|,-strand,0-1542 chimera ENST00000582467.2|ENSG000001716 34.19|OTTHUMG00000132254.6|OTTHUMT00000448091.2|BPTF-215|BPTF|11789|protein_coding|,+strand,0-11789 length=12759 error-free_length=13347 read_identity=87.51%).
# qualities=12894, # bases=12759
at stream.Read.validate(Read.java:114)
at stream.Read.<init>(Read.java:78)
at stream.Read.<init>(Read.java:61)
at stream.FASTQ.quadToRead(FASTQ.java:872)
at stream.FASTQ.toReadList(FASTQ.java:705)
at stream.FastqReadInputStream.fillBuffer(FastqReadInputStream.java:111)
at stream.FastqReadInputStream.nextList(FastqReadInputStream.java:96)
at stream.ConcurrentGenericReadInputStream$ReadThread.readLists(ConcurrentGenericReadInputStream.java:656)
at stream.ConcurrentGenericReadInputStream$ReadThread.run(ConcurrentGenericReadInputStream.java:635)
Input: 1934000 reads 3369013255 bases
Output: 1934000 reads (100.00%) 3369013255 bases (100.00%)
Time: 50.653 seconds.
Reads Processed: 1934k 38.18k reads/sec
Bases Processed: 3369m 66.51m bases/sec
Exception in thread "main" java.lang.RuntimeException: ReformatReads terminated in an error state; the output may be corrupt.
at jgi.ReformatReads.process(ReformatReads.java:1103)
at jgi.ReformatReads.main(ReformatReads.java:43)
ERROR: Command failed with exit status = 1 :
/opt/JAFFA-version-2.3/tools/bin/reformat ignorebadquality=t in=../positive_control_nanopore2020_D20x_M85.fastq.gz out=positive_control_nanopore2020_D20x_M85.fastq/positive_control_nanopore2020_D20x_M85.fastq.fasta threads=1 ;
================ Stage filter_transcripts (positive_control_nanopore2020_D10x_M95) =================
java -ea -Xmx200m -cp /opt/JAFFA-version-2.3/tools/bbmap/current/ jgi.ReformatReads ignorebadquality=t in=../positive_control_nanopore2020_D20x_M85.fastq.gz out=positive_control_nanopore2020_D20x_M85.fastq/positive_control_nanopore2020_D2 0x_M85.fastq.fasta threads=1
Executing jgi.ReformatReads [ignorebadquality=t, in=../positive_control_nanopore2020_D20x_M85.fastq.gz, out=positive_control_nanopore2020_D20x_M85.fastq/positive_control_nanopore2020_D20x_M85.fastq.fasta, threads=1]
Set threads to 1
Input is being processed as unpaired
[M::worker_pipeline::3857.303*1.00] mapped 286876 sequences
java.lang.AssertionError:
Mismatch between length of bases and qualities for read 1934044 (id=d642ebd4-a024-95da-520b-1b29b14a4348 ENST00000361624.2|ENSG00000198804.2|-|-|MT-CO1-201|MT-CO1|1542|protein_coding|,-strand,0-1542 chimera ENST00000582467.2|ENSG000001716 34.19|OTTHUMG00000132254.6|OTTHUMT00000448091.2|BPTF-215|BPTF|11789|protein_coding|,+strand,0-11789 length=12759 error-free_length=13347 read_identity=87.51%).
# qualities=12894, # bases=12759
......
at stream.Read.validate(Read.java:114)
at stream.Read.<init>(Read.java:78)
at stream.Read.<init>(Read.java:61)
at stream.FASTQ.quadToRead(FASTQ.java:872)
at stream.FASTQ.toReadList(FASTQ.java:705)
at stream.FastqReadInputStream.fillBuffer(FastqReadInputStream.java:111)
at stream.FastqReadInputStream.nextList(FastqReadInputStream.java:96)
at stream.ConcurrentGenericReadInputStream$ReadThread.readLists(ConcurrentGenericReadInputStream.java:656)
at stream.ConcurrentGenericReadInputStream$ReadThread.run(ConcurrentGenericReadInputStream.java:635)
Input: 1934000 reads 3369013255 bases
Output: 1934000 reads (100.00%) 3369013255 bases (100.00%)
Time: 50.653 seconds.
Reads Processed: 1934k 38.18k reads/sec
Bases Processed: 3369m 66.51m bases/sec
Exception in thread "main" java.lang.RuntimeException: ReformatReads terminated in an error state; the output may be corrupt.
at jgi.ReformatReads.process(ReformatReads.java:1103)
at jgi.ReformatReads.main(ReformatReads.java:43)
ERROR: Command failed with exit status = 1 :
/opt/JAFFA-version-2.3/tools/bin/reformat ignorebadquality=t in=../positive_control_nanopore2020_D20x_M85.fastq.gz out=positive_control_nanopore2020_D20x_M85.fastq/positive_control_nanopore2020_D20x_M85.fastq.fasta threads=1 ;
Despite these errors, the files will continue into later stages:
============= Stage extract_fusion_sequences (positive_control_nanopore2020_D10x_M95) ==============
java -ea -Xmx200m -cp /opt/JAFFA-version-2.3/tools/bbmap/current/ jgi.ReformatReads ignorebadquality=t in=../positive_control_nanopore2020_D20x_M90.fastq.gz out=positive_control_nanopore2020_D20x_M90.fastq/positive_control_nanopore2020_D20x_M90.fastq.fasta threads=1
Executing jgi.ReformatReads [ignorebadquality=t, in=../positive_control_nanopore2020_D20x_M90.fastq.gz, out=positive_control_nanopore2020_D20x_M90.fastq/positive_control_nanopore2020_D20x_M90.fastq.fasta, threads=1]
Set threads to 1
Input is being processed as unpaired
[M::worker_pipeline::4170.854*1.00] mapped 207573 sequences
[M::main] Version: 2.17-r941
[M::main] CMD: /opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x map-ont -c /opt/JAFFA-version-2.3/hg38_genCode22.fa positive_control_nanopore2020_D10x_M90.fastq/positive_control_nanopore2020_D10x_M90.fastq.fasta
[M::main] Real time: 4170.911 sec; CPU: 4170.349 sec; Peak RSS: 2.835 GB
================ Stage filter_transcripts (positive_control_nanopore2020_D10x_M90) =================
Done reading in transcript IDs
Reading the input alignment file, jaffal_edited_positive_control_nanopore2020_D100x_M85.fastq/jaffal_edited_positive_control_nanopore2020_D100x_M85.fastq.paf
0
1000000
2000000
3000000
3038753 reads processed. Finished.
java -ea -Xmx200m -cp /opt/JAFFA-version-2.3/tools/bbmap/current/ jgi.ReformatReads in=jaffal_edited_positive_control_nanopore2020_D10x_M85.fastq/jaffal_edited_positive_control_nanopore2020_D10x_M85.fastq.fasta out=stdout.fasta fastawrap=0
....
============= Stage extract_fusion_sequences (positive_control_nanopore2020_D10x_M90) ==============
Executing jgi.ReformatReads [in=jaffal_edited_positive_control_nanopore2020_D100x_M85.fastq/jaffal_edited_positive_control_nanopore2020_D100x_M85.fastq.fasta, out=stdout.fasta, fastawrap=0]
Input is being processed as unpaired
Input: 2418119 reads 4376157978 bases
Output: 2418119 reads (100.00%) 4376157978 bases (100.00%)
Time: 74.634 seconds.
Reads Processed: 2418k 32.40k reads/sec
Bases Processed: 4376m 58.64m bases/sec
================== Stage minimap2_genome (positive_control_nanopore2020_D10x_M95) ==================
[M::mm_idx_gen::11.980*0.83] collected minimizers
[M::mm_idx_gen::16.631*0.88] sorted minimizers
[M::main::16.631*0.88] loaded/built the index for 195175 target sequence(s)
[M::mm_mapopt_update::17.107*0.88] mid_occ = 110
[M::mm_idx_stat] kmer size: 15; skip: 10; is_hpc: 0; #seq: 195175
[M::mm_idx_stat::17.405*0.88] distinct minimizers: 16950829 (43.60% are singletons); average occurrences: 3.243; average spacing: 5.406
[M::worker_pipeline::3316.073*1.00] mapped 276210 sequences
Input: 3432565 reads 6048616768 bases
Output: 3432565 reads (100.00%) 6048616768 bases (100.00%)
Time: 104.254 seconds.
Reads Processed: 3432k 32.92k reads/sec
Bases Processed: 6048m 58.02m bases/sec
================== Stage minimap2_genome (positive_control_nanopore2020_D10x_M90) ==================
[M::mm_idx_gen::129.049*0.96] collected minimizers
[M::worker_pipeline::961.815*1.00] mapped 282471 sequences
[M::worker_pipeline::4260.013*1.00] mapped 276724 sequences
[M::mm_idx_gen::217.976*0.98] sorted minimizers
[M::main::217.976*0.98] loaded/built the index for 455 target sequence(s)
[M::mm_mapopt_update::221.762*0.98] mid_occ = 792
[M::mm_idx_stat] kmer size: 15; skip: 5; is_hpc: 0; #seq: 455
[M::mm_idx_stat::224.110*0.98] distinct minimizers: 167291034 (34.68% are singletons); average occurrences: 6.239; average spacing: 3.075
[M::worker_pipeline::1435.168*1.00] mapped 281994 sequences
[M::worker_pipeline::4730.434*1.00] mapped 276264 sequences
[M::worker_pipeline::1019.761*0.99] mapped 12621 sequences
[M::main] Version: 2.17-r941
[M::main] CMD: /opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa positive_control_nanopore2020_D10x_M95.fastq/positive_control_nanopore2020_D10x_M95.fastq.fusions.fa
[M::main] Real time: 1019.905 sec; CPU: 1014.720 sec; Peak RSS: 18.983 GB
============== Stage make_fasta_reads_table (positive_control_nanopore2020_D10x_M95) ===============
[M::worker_pipeline::1907.097*1.00] mapped 282043 sequences
[M::worker_pipeline::5201.252*1.00] mapped 276573 sequences
[M::mm_idx_gen::144.855*0.86] collected minimizers
[M::mm_idx_gen::233.550*0.91] sorted minimizers
[M::main::233.550*0.91] loaded/built the index for 455 target sequence(s)
[M::mm_mapopt_update::237.213*0.92] mid_occ = 792
[M::mm_idx_stat] kmer size: 15; skip: 5; is_hpc: 0; #seq: 455
[M::mm_idx_stat::239.559*0.92] distinct minimizers: 167291034 (34.68% are singletons); average occurrences: 6.239; average spacing: 3.075
[M::worker_pipeline::2381.133*1.00] mapped 281944 sequences
[M::worker_pipeline::5674.755*1.00] mapped 276818 sequences
[M::worker_pipeline::2854.149*1.00] mapped 281181 sequences
[M::worker_pipeline::6145.728*1.00] mapped 276402 sequences
[M::worker_pipeline::3327.070*1.00] mapped 281324 sequences
[M::worker_pipeline::6616.826*1.00] mapped 276139 sequences
[M::worker_pipeline::3799.048*1.00] mapped 282279 sequences
[M::worker_pipeline::7087.283*1.00] mapped 275430 sequences
[M::worker_pipeline::2286.720*0.99] mapped 25347 sequences
[M::main] Version: 2.17-r941
[M::main] CMD: /opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa jaffal_edited_positive_control_nanopore2020_D100x_M85.fastq/jaffal_edited_positive_control_nanopore2020_D100x_M85.fastq.fusions.fa
[M::main] Real time: 2286.867 sec; CPU: 2266.908 sec; Peak RSS: 18.982 GB
============== Stage make_fasta_reads_table (positive_control_nanopore2020_D10x_M90) ===============
================== Stage get_final_list (positive_control_nanopore2020_D10x_M95) ===================
================== Stage get_final_list (positive_control_nanopore2020_D10x_M90) ===================
R version 4.3.0 (2023-04-21) -- "Already Tomorrow"
Copyright (C) 2023 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)
R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.
Natural language support but running in an English locale
R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.
Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.
> options(echo=FALSE)
[1] "Getting the location of fusion transcripts in the genome.."
Calculating gap size in the genome...
Checking if the fusions are in frame...
[M::worker_pipeline::6645.681*1.00] mapped 281746 sequences
[M::worker_pipeline::9932.681*1.00] mapped 276604 sequences
Merging with read coverage data...
Reassigning Low Confidence breakpoints
Done producing summary file
=============== Stage report_3_gene_fusions (positive_control_nanopore2020_D10x_M95) ===============
read id: f0b35753-71f3-f9c3-9a50-77dbb994040b does not conform to 3 gene fusion structure.
However I then get the same error message again in
== Stage report_3_gene_fusions (positive_control_nanopore2020_D10x_M90) ==
java -ea -Xmx200m -cp /opt/JAFFA-version-2.3/tools/bbmap/current/ jgi.ReformatReads ignorebadquality=t in=../positive_control_nanopore2020_D50x_M85.fastq.gz out=positive_control_nanopore2020_D50x_M85.fastq/positive_control_nanopore2020_D50x_M85.fastq.fasta threads=1
Executing jgi.ReformatReads [ignorebadquality=t, in=../positive_control_nanopore2020_D50x_M85.fastq.gz, out=positive_control_nanopore2020_D50x_M85.fastq/positive_control_nanopore2020_D50x_M85.fastq.fasta, threads=1]
Set threads to 1
Input is being processed as unpaired
java.lang.AssertionError:
Mismatch between length of bases and qualities for read 2524899 (id=8f66a3fe-c408-fef8-1779-6ce91cc4a34f ENST00000344063.7|ENSG00000108107.15|OTTHUMG00000171991.4|OTTHUMT00000416277.2|RPL28-201|RPL28|4209|protein_coding|,+strand,0-4209 length=4170 error-free_length=4235 read_identity=94.79%).
# qualities=135, # bases=4170
@c9990bd9-8607-294c-8734-feb361710e64 ref|NM_018666-NM_001330000,+strand,0-5484 length=5419 error-free_length=5485 read_identity=89.20%
CGTTCAGTTACGTATTGCTCTCTTTCCGGTCATCAGGTCGCCGCTGCGAAGGGGCCGCCACCATGTCTGCGCATCTGCAATGGATGGTCGTGCGGAACTGCTCCAGTTTCCTGATCAACGAGGAATAAGCAGACCTACAGCACTGAGCCCAATAACTTGAAGGCCCGCAATTCAACTTCCGCTACAGCGGATGATTCACGCAGGACTGTGGGCGTGGAGCCGGCGGCCGACGGCAAAGTGTCGTGGTGGTCATTAAGCAGATCCGGCCAGCGGAAGCCTGCCACCTCCTATGTGCGGACACCATCAACAAGAATGCTCGCGCCCACGCTCAGCAGCATCAGACACATGATCCGCAAGAACAAGTACCGCCCCGACCTGCTGATGGCAGCCATCCGCAGGGCCAGCGCCATCCTGTGCAGCCAGAAGCCTGTGATGGTGAAGAAAGCGGACCGCCCCACCAAGAGCTCCTGAGCCCCCTGCCCCCAGAGCAATAAAGTCAGCTGCTTTCTCACCCTGCCTCGACGGGCCTCCCTTTTTTGAAACGCTCTGGGGAGCTCTGGCCTATGTTGTCATTCAGGCCATGTCATCAAACTCTGCATGTCACCTTGTCCATCTGGAGGTGATGTCAATGGCTGGCCACATGCAGGAGGGGTGGGGTAGCTGCCTTGTCCTTTGGTGGCAAGGGTCACTGTCTTCACAGAAAAATTTGCTGACTTGTGATTGAACCTACTGTCCCATTGTGAGGTGGCCTGAAGAATCCCAGCTGGGGAGTGGCTTCCATTCAGAAGAAGAAAGGCCTTTTCTAGCCCAGAAGGGTGCAGGCTGAGGCTGGGCCCTGGGCCCTGGTGCTGTAGCACGGTTTGGGGACTTGGGGTGTTCCCAAGACCTGGGGGACGACGAACATCCGGGAGGAAGATGAGAGACTTTTGCATCCAGGAGTGGGTGCGGCCACATTTGGAGGGGATGGAGCTTTGATGCAACCTGTCTCTGAGATGGGCAACTTTGGTGGGTGGTGGCTTATAACTGTAAGGGGATGGCTGACCAGGGTACAGCCAGCAGGCATTGAGCAGCCTTAGCATTGTCCCCCTACTCCCGTCCTCCAGGTGTCCGCATCCCTCCCCTATCTCTTTGAGCTGGCTCTTGTCACCTTAGGACTCATCTAGTGGCCGCTCTTTGGGCCACCCTGTCACCCAAGCTTTCCTGATTGCCCAGCCCTCTTGTTTCCTTTGGCCTGTTTGCTCCCTAGTGTTTATTACAGCTTGTGAGGCCAGGAGTTTGAGACCATCCTAGGCAACACCAATGAGACACCGTCTCTAAAATAAAATTAGCTGGGTGTGGTGGTCACCGCCTGTGGTCCCAGCTCCTCAGAGGTTGAGTGGGGCTGAGGGACGGAGCACTTGAACCAAGAATATGGGGCTGCAGTAGCCCAGAGCCCCACCACCACACTCCAGCCTGGAAGACACCATGACACAGTGAGGCCTGGATGGAAAAGAGTCCTGTGTTGATCCTCACATGTTTCCTGAACACTAACTCTGTCAGCCACTGCCAGGACCAAAGGATCAGCATCATGGCACCCCTGGTTCCACTGCCATCCTGGGGGTACCGATTCAAAGAAGGACTCTGCTCCCTGTCTGAGACCACCCCCCGGCTCTGACTGAGATAAGGGGACTGTCAGGGCCTCGACTTGCCATTGGTTGGGGTCGTACGGGGCTGGGAGCCTGCGTTTTGAGGCGGGCCTGCCCTTCCGACCTCAGTCCTGTCTGCTCCAGTCTTGCCCAACTCGAAGGGAGCAGATCTGACCACTTGCCAGCCCCTGTCTGCTGTGAATTACATTTCCTTTGTCTTCCTTAGTTGGGTCTATTAGCTCAGATTGAGGTGTTACCTTAAAATGAGTTGGGTGACTTGGTACCTGCTCAGGACTCCCCCGCACTGTCCGTCCCACTCAGGCCCACCTCCAGCTGGCCTCACTCCGCTGAGTGACTTCGTACCTGCTCAGGAGCCCCCACTGTCCCAGTCCCACTCAGGCCCATCTCGCTGGCCTCACTGCGCTGGGACTCCGCCTTCATAAAGGAGAGCTCACTGCTCACGTTAGTAAGATGGCCCCTTCTCGTGAGGCCTCTCCCTGGCACCTGCTTCAGTTGTCCTCCACAGCACTGATTTGCAGCCCACAAGCTGGCAGGTTTATCTGTCTCATGTTTGTCTTGTGCTGGTCGGCAAGGGGTTTAGTCTAGCACACCAGCATATAATGAGATGCTTGATGAATGGTACATATTGAATGTATAAAGCCCACCGGTCCTGAGTTTGCTCACTGGAGACTTTCTGGAGATGGAGTCTCGCTCTGTGCCCGGCTGGCGAAGTGCAATGGCGCGATCTTGGCTCACTGCAGCCTCCCACCTCCTGGGTTCAACGATTCTCCTGCCTCAGCCTCCCGAGTAGCTGGGATTACAGGTGGTATGCCACACCCAGCTCAGTATTGTATTTTTAGCAGAGATCGGGTTTCACCATTGCAGGCTGGTTTGGAACTCCTGACTTCAAATTACCCACCTGCCTCAACCTCCCAAAGTGCTGGCATTACAAGCGCTCGAGGCTTCTGATGTGGCTGCTGCTGCTCAGAAGGCCTTGTCCTTAACCACCTCCTTGCCTGCCCTGGAGGCTTGTGCCTCTAGGACCCCACCCCTGTGGAGTCCTGCTGGCTTTCTCCATCCCTATCTGAATCCTCCTGCTGTGTGGCCTCCCCTGGTCTCATCCGTAACACAGCCCAACAGTGGCCTCTGTTCCTGCGGGTGGCCAGCCCATCTGTGTGTGGCTGGGCTGGGGAGGCCACGCGTCTGGTATCTGAATGCTATCGGTGGGTTGGGGTGGAGGAACCAGGAGAGGGCTGGAGGGAGAGGAGATGGTCTCAGCCCCACAGAGTTTGGATGTCATCAGTGTGCTGAGCAAACGTGAACACCATTTCCCTCCTCTAGACCTCATCTTGGAGAGAGAGATGTTGGATGGGGCCATCTATTCCAGCTTTATTCACAAATCATGTCTGTTGGCCTGGAAATTGGAAAACCAGTTAAACCAAAAACGTGATATTAAGAAAACAGGCGGAGCTCACCATTGTAAAAATGCTGAAAGCCAAAGACAAAGTGGAGAACAAAGAAAAGCGTCTTGTCACATTACAGAAGGTCCTGATAAAGTAGTAGCTGCCCTCATCTAGAAACCAGGCCCAGGCAGTGGGGACACATCCAGTGCTGAAAGAACCTCCCCCAGGTCATCTATCCCAAGAGTGATGCCCGGCAGCATTCCAAGCTCAAGGGCTAATGGTTCACGGAAGCCAGGAATCAAACTGCCTGGGTTCCAGTCCCAGCTCTGCCAGTTATGCCCAGCTGTGGGACTTGGGCAACTCGTTTAGTAGCACCGTGCCTCAGTTCCCAATATGTAAAAGGCCATTTGAGTGCCTTTCACAGCCCTGCATAAGGCAGGTGTCTCAGTGTTCACTGCTATCTCTCCAGCTCTTAGTCCAGTAGCTACATGGTGAGTGAGCGTAGGGCGCACCCTGGAAGGCTGCCCAGCCCAAAGTTGTGCCAGAGCGCTGGGGATCAGACTCCCCACAGCAGCAGAGACTCAGGGACTGAGGCATCCTCTGTCACAGGACATGCTGCGTCTACTGGGTCAGGGCTCTGCTGCTCGGTGGCTGTGAACCTTAGGCAAGTTCCTCAACCTCTCTGTGTCTTCGTACCCTCATCTGTAACATGCGTGTCGATAGACCCTACTACTCCGGGTTGGTGAAGATTAAATGTGCAAAACCTGCTTGACGCTGTGCCCACAAATCCTGATTGTAGGAATAAATTAATGACTTTTTATAAATATTTTGATCGATGGACTCATGATCTGGATGTCTCACATGCCAATGACTAATTTGTACACAAACTAATGCTCGTGTTTCCCGAGCACCTGGAGAGACATAGCCAGATCCATGTGCGATAATGCCTGGTGGCTCCAGGTTGCCCCGCCGTCCTGTGGGCTGTGAGCTTTCCCAGCCTCCTGCCCGTGTTTGTGAATATCATTCTGTCCTCCAGCTGCATTTCCAGCCGAGCTCTTTGGCGCTGCCCAGGAATGGTATCAATTCCCCTGTTTCTCTGTAGCCAGTTACTAGAATAAAATCATCTACTTTAAAAGCAATAC
at stream.Read.validate(Read.java:114)
at stream.Read.<init>(Read.java:78)
at stream.Read.<init>(Read.java:61)
at stream.FASTQ.quadToRead(FASTQ.java:872)
at stream.FASTQ.toReadList(FASTQ.java:705)
at stream.FastqReadInputStream.fillBuffer(FastqReadInputStream.java:111)
at stream.FastqReadInputStream.nextList(FastqReadInputStream.java:96)
at stream.ConcurrentGenericReadInputStream$ReadThread.readLists(ConcurrentGenericReadInputStream.java:656)
at stream.ConcurrentGenericReadInputStream$ReadThread.run(ConcurrentGenericReadInputStream.java:635)
Input: 2524800 reads 4397913090 bases
Output: 2524800 reads (100.00%) 4397913090 bases (100.00%)
Time: 66.580 seconds.
Reads Processed: 2524k 37.92k reads/sec
Bases Processed: 4397m 66.05m bases/sec
Exception in thread "main" java.lang.RuntimeException: ReformatReads terminated in an error state; the output may be corrupt.
at jgi.ReformatReads.process(ReformatReads.java:1103)
at jgi.ReformatReads.main(ReformatReads.java:43)
read id: 97bbcd5f-5c99-c079-08c5-f9de492e7062 does not conform to 3 gene fusion structure.
read id: 0ed90292-a03b-79f8-49c7-a0f6cad8a986 does not conform to 3 gene fusion structure.
read id: a8a332ef-bc95-08a1-ff04-adc08e25bde0 does not conform to 3 gene fusion structure.
read id: 785523d8-f840-1378-82ab-088fd95db875 does not conform to 3 gene fusion structure.
read id: 7ed251aa-ab58-b50a-fab5-1d65e4bc61f6 does not conform to 3 gene fusion structure.
read id: 13001a1f-1ef1-115e-16e8-6b80f23c960c does not conform to 3 gene fusion structure.
read id: 26a7d451-1849-ee01-4ef4-7e5b4f9731b4 does not conform to 3 gene fusion structure.
java -ea -Xmx200m -cp /opt/JAFFA-version-2.3/tools/bbmap/current/ jgi.ReformatReads ignorebadquality=t in=../positive_control_nanopore2020_D20x_M95.fastq.gz out=positive_control_nanopore2020_D20x_M95.fastq/positive_control_nanopore2020_D20x_M95.fastq.fasta threads=1
Running JAFFAL on only the spiked fusions, and only the background simulated reads
In an attempt to narrow down what may be cause the errors, I tried run JAFFAL on on only the spiked fusions, and only the background simulated reads (these files are appended to each other to make my "positive_control" data files I ran JAFFAL on earlier. The pipeline fails, but with different exit statuses and error messages background reads, no spiked fusions
========================================= Pipeline Failed ==========================================
One or more parallel stages aborted. The following messages were reported:
Branch negative.control.nanopore2020_D10x_M90_HepG2.fastq in stage Unknown reported message:
Command failed with exit status = 137 :
/opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa negative.control.nanopore2020_D10x_M90_HepG2.fastq/negative.control.nanopore2020_D10x_M90_HepG2.fastq.fusions.fa > negative.control.nanopore2020_D10x_M90_HepG2.fastq/negative.control.nanopore2020_D10x_M90_HepG2.fastq_genome.paf; grep $'\t+\t' negative.control.nanopore2020_D10x_M90_HepG2.fastq/negative.control.nanopore2020_D10x_M90_HepG2.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",",$3","$3+100",", $8","$9-$4+$3+100"," }' > negative.control.nanopore2020_D10x_M90_HepG2.fastq/negative.control.nanopore2020_D10x_M90_HepG2.fastq_genome.psl ; grep $'\t-\t' negative.control.nanopore2020_D10x_M90_HepG2.fastq/negative.control.nanopore2020_D10x_M90_HepG2.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",", $2-$4","$2-$4+100",", $8","$9-$4+$3+100"," }' >> negative.control.nanopore2020_D10x_M90_HepG2.fastq/negative.control.nanopore2020_D10x_M90_HepG2.fastq_genome.psl ;
Branch negative.control.nanopore2020_D10x_M95_HepG2.fastq in stage Unknown reported message:
Command failed with exit status = 137 :
/opt/JAFFA-version-2.3/tools/bin/minimap2 -t 5 -x splice -c /opt/JAFFA-version-2.3/hg38.fa negative.control.nanopore2020_D10x_M95_HepG2.fastq/negative.control.nanopore2020_D10x_M95_HepG2.fastq.fusions.fa > negative.control.nanopore2020_D10x_M95_HepG2.fastq/negative.control.nanopore2020_D10x_M95_HepG2.fastq_genome.paf; grep $'\t+\t' negative.control.nanopore2020_D10x_M95_HepG2.fastq/negative.control.nanopore2020_D10x_M95_HepG2.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",",$3","$3+100",", $8","$9-$4+$3+100"," }' > negative.control.nanopore2020_D10x_M95_HepG2.fastq/negative.control.nanopore2020_D10x_M95_HepG2.fastq_genome.psl ; grep $'\t-\t' negative.control.nanopore2020_D10x_M95_HepG2.fastq/negative.control.nanopore2020_D10x_M95_HepG2.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",", $2-$4","$2-$4+100",", $8","$9-$4+$3+100"," }' >> negative.control.nanopore2020_D10x_M95_HepG2.fastq/negative.control.nanopore2020_D10x_M95_HepG2.fastq_genome.psl ;
Use 'bpipe errors' to see output from failed commands.
ryleyd@minion:/data2/ryleyd/benchmark_simulated_reads/no_fusions/troubleshooting$ /opt/JAFFA-version-2.3/tools/bin/bpipe errors
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.codehaus.groovy.reflection.CachedClass$3$1 (file:/opt/JAFFA-version-2.3/tools/bpipe-0.9.9.2/lib/bpipe.jar) to method java.lang.Object.finalize()
WARNING: Please consider reporting this to the maintainers of org.codehaus.groovy.reflection.CachedClass$3$1
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
============================= Found 2 failed commands from run 488499 ==============================
============================================ Command 24 ============================================
Command : /opt/JAFFA-version-2.3/tools/bin/minimap2 -t 5 -x splice -c /opt/JAFFA-version-2.3/hg38.fa negative.control.nanopore2020_D10x_M95_HepG2.fastq/negative.control.nanopore2020_D10x_M95_HepG2.fastq.fusions.fa > negative.control.nanopore2020_D10x_M95_HepG2.fastq/negative.control.nanopore2020_D10x_M95_HepG2.fastq_genome.paf; grep $'\t+\t' negative.control.nanopore2020_D10x_M95_HepG2.fastq/negative.control.nanopore2020_D10x_M95_HepG2.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",",$3","$3+100",", $8","$9-$4+$3+100"," }' > negative.control.nanopore2020_D10x_M95_HepG2.fastq/negative.control.nanopore2020_D10x_M95_HepG2.fastq_genome.psl ; grep $'\t-\t' negative.control.nanopore2020_D10x_M95_HepG2.fastq/negative.control.nanopore2020_D10x_M95_HepG2.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",", $2-$4","$2-$4+100",", $8","$9-$4+$3+100"," }' >> negative.control.nanopore2020_D10x_M95_HepG2.fastq/negative.control.nanopore2020_D10x_M95_HepG2.fastq_genome.psl ;
Exit Code : 137
Output :
No command output found in most recent log file
============================================ Command 25 ============================================
Command : /opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa negative.control.nanopore2020_D10x_M90_HepG2.fastq/negative.control.nanopore2020_D10x_M90_HepG2.fastq.fusions.fa > negative.control.nanopore2020_D10x_M90_HepG2.fastq/negative.control.nanopore2020_D10x_M90_HepG2.fastq_genome.paf; grep $'\t+\t' negative.control.nanopore2020_D10x_M90_HepG2.fastq/negative.control.nanopore2020_D10x_M90_HepG2.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",",$3","$3+100",", $8","$9-$4+$3+100"," }' > negative.control.nanopore2020_D10x_M90_HepG2.fastq/negative.control.nanopore2020_D10x_M90_HepG2.fastq_genome.psl ; grep $'\t-\t' negative.control.nanopore2020_D10x_M90_HepG2.fastq/negative.control.nanopore2020_D10x_M90_HepG2.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",", $2-$4","$2-$4+100",", $8","$9-$4+$3+100"," }' >> negative.control.nanopore2020_D10x_M90_HepG2.fastq/negative.control.nanopore2020_D10x_M90_HepG2.fastq_genome.psl ;
Exit Code : 137
Output :
No command output found in most recent log file
Other errors when JAFFAL is run only on background reads
========== Stage extract_fusion_sequences (negative.control.nanopore2020_D10x_M85_HepG2) ===========
bash: line 1: 491905 Killed /opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa negative.control.nanopore2020_D10x_M90_HepG2.fastq/negative.control.nanopore2020_D10x_M90_HepG2.fastq.fusions.fa > negative.control.nanopore2020_D10x_M90_HepG2.fastq/negative.control.nanopore2020_D10x_M90_HepG2.fastq_genome.paf
java -ea -Xmx200m -cp /opt/JAFFA-version-2.3/tools/bbmap/current/ jgi.ReformatReads in=negative.control.nanopore2020_D10x_M85_HepG2.fastq/negative.control.nanopore2020_D10x_M85_HepG2.fastq.fasta out=stdout.fasta fastawrap=0
Executing jgi.ReformatReads [in=negative.control.nanopore2020_D10x_M85_HepG2.fastq/negative.control.nanopore2020_D10x_M85_HepG2.fastq.fasta, out=stdout.fasta, fastawrap=0]
Input is being processed as unpaired
ERROR: Command failed with exit status = 137 :
/opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa negative.control.nanopore2020_D10x_M90_HepG2.fastq/negative.control.nanopore2020_D10x_M90_HepG2.fastq.fusions.fa > negative.control.nanopore2020_D10x_M90_HepG2.fastq/negative.control.nanopore2020_D10x_M90_HepG2.fastq_genome.paf; grep $'\t+\t' negative.control.nanopore2020_D10x_M90_HepG2.fastq/negative.control.nanopore2020_D10x_M90_HepG2.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",",$3","$3+100",", $8","$9-$4+$3+100"," }' > negative.control.nanopore2020_D10x_M90_HepG2.fastq/negative.control.nanopore2020_D10x_M90_HepG2.fastq_genome.psl ; grep $'\t-\t' negative.control.nanopore2020_D10x_M90_HepG2.fastq/negative.control.nanopore2020_D10x_M90_HepG2.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",", $2-$4","$2-$4+100",", $8","$9-$4+$3+100"," }' >> negative.control.nanopore2020_D10x_M90_HepG2.fastq/negative.control.nanopore2020_D10x_M90_HepG2.fastq_genome.psl ;
Input: 1727872 reads 3009507128 bases
Output: 1727872 reads (100.00%) 3009507128 bases (100.00%)
Time: 50.819 seconds.
Reads Processed: 1727k 34.00k reads/sec
Bases Processed: 3009m 59.22m bases/sec
=============== Stage minimap2_genome (negative.control.nanopore2020_D10x_M85_HepG2) ===============
[M::mm_idx_gen::781.869*0.20] collected minimizers
bash: line 1: 491895 Killed /opt/JAFFA-version-2.3/tools/bin/minimap2 -t 5 -x splice -c /opt/JAFFA-version-2.3/hg38.fa negative.control.nanopore2020_D10x_M95_HepG2.fastq/negative.control.nanopore2020_D10x_M95_HepG2.fastq.fusions.fa > negative.control.nanopore2020_D10x_M95_HepG2.fastq/negative.control.nanopore2020_D10x_M95_HepG2.fastq_genome.paf
ERROR: Command failed with exit status = 137 :
/opt/JAFFA-version-2.3/tools/bin/minimap2 -t 5 -x splice -c /opt/JAFFA-version-2.3/hg38.fa negative.control.nanopore2020_D10x_M95_HepG2.fastq/negative.control.nanopore2020_D10x_M95_HepG2.fastq.fusions.fa > negative.control.nanopore2020_D10x_M95_HepG2.fastq/negative.control.nanopore2020_D10x_M95_HepG2.fastq_genome.paf; grep $'\t+\t' negative.control.nanopore2020_D10x_M95_HepG2.fastq/negative.control.nanopore2020_D10x_M95_HepG2.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",",$3","$3+100",", $8","$9-$4+$3+100"," }' > negative.control.nanopore2020_D10x_M95_HepG2.fastq/negative.control.nanopore2020_D10x_M95_HepG2.fastq_genome.psl ; grep $'\t-\t' negative.control.nanopore2020_D10x_M95_HepG2.fastq/negative.control.nanopore2020_D10x_M95_HepG2.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",", $2-$4","$2-$4+100",", $8","$9-$4+$3+100"," }' >> negative.control.nanopore2020_D10x_M95_HepG2.fastq/negative.control.nanopore2020_D10x_M95_HepG2.fastq_genome.psl ;
[M::mm_idx_gen::701.496*0.21] collected minimizers
[M::mm_idx_gen::1218.171*0.29] sorted minimizers
[M::main::1218.171*0.29] loaded/built the index for 455 target sequence(s)
[M::mm_mapopt_update::1221.510*0.29] mid_occ = 792
[M::mm_idx_stat] kmer size: 15; skip: 5; is_hpc: 0; #seq: 455
[M::mm_idx_stat::1223.453*0.29] distinct minimizers: 167291034 (34.68% are singletons); average occurrences: 6.239; average spacing: 3.075
[M::worker_pipeline::1414.630*0.39] mapped 4878 sequences
[M::main] Version: 2.17-r941
[M::main] CMD: /opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa negative.control.nanopore2020_D10x_M85_HepG2.fastq/negative.control.nanopore2020_D10x_M85_HepG2.fastq.fusions.fa
[M::main] Real time: 1414.818 sec; CPU: 549.583 sec; Peak RSS: 18.681 GB
Only fusion reads, no background reads
========================================= Pipeline Failed ==========================================
One or more parallel stages aborted. The following messages were reported:
Branch nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq in stage Unknown reported message:
Command failed with exit status = 137 :
/opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq/nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq.fusions.fa > nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq/nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq_genome.paf; grep $'\t+\t' nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq/nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",",$3","$3+100",", $8","$9-$4+$3+100"," }' > nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq/nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq_genome.psl ; grep $'\t-\t' nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq/nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",", $2-$4","$2-$4+100",", $8","$9-$4+$3+100"," }' >> nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq/nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq_genome.psl ;
Use 'bpipe errors' to see output from failed commands.
ryleyd@minion:/data2/ryleyd/benchmark_simulated_reads/fusions/JAFFAL_troubleshooting$ /opt/JAFFA-version-2.3/tools/bin/bpipe errors
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.codehaus.groovy.reflection.CachedClass$3$1 (file:/opt/JAFFA-version-2.3/tools/bpipe-0.9.9.2/lib/bpipe.jar) to method java.lang.Object.finalize()
WARNING: Please consider reporting this to the maintainers of org.codehaus.groovy.reflection.CachedClass$3$1
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
============================= Found 1 failed commands from run 488798 ==============================
============================================ Command 27 ============================================
Command : /opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq/nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq.fusions.fa > nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq/nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq_genome.paf; grep $'\t+\t' nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq/nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",",$3","$3+100",", $8","$9-$4+$3+100"," }' > nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq/nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq_genome.psl ; grep $'\t-\t' nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq/nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",", $2-$4","$2-$4+100",", $8","$9-$4+$3+100"," }' >> nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq/nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq_genome.psl ;
Exit Code : 137
Output :
No command output found in most recent log file
However, the only error that appears while running the code is as below:
============== Stage minimap2_genome (nanopore2020_D10x_M85_benchmark_fusions.fasta) ===============
[M::mm_idx_gen::86.966*1.57] collected minimizers
[M::mm_idx_gen::272.588*0.52] collected minimizers
[M::mm_idx_gen::412.544*0.35] collected minimizers
[M::mm_idx_gen::2049.733*0.13] sorted minimizers
[M::main::2049.754*0.13] loaded/built the index for 455 target sequence(s)
[M::mm_mapopt_update::2131.577*0.12] mid_occ = 792
[M::mm_idx_stat] kmer size: 15; skip: 5; is_hpc: 0; #seq: 455
[M::mm_idx_stat::2133.926*0.12] distinct minimizers: 167291034 (34.68% are singletons); average occurrences: 6.239; average spacing: 3.075
bash: line 1: 489181 Killed /opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq/nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq.fusions.fa > nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq/nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq_genome.paf
ERROR: Command failed with exit status = 137 :
/opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq/nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq.fusions.fa > nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq/nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq_genome.paf; grep $'\t+\t' nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq/nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",",$3","$3+100",", $8","$9-$4+$3+100"," }' > nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq/nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq_genome.psl ; grep $'\t-\t' nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq/nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",", $2-$4","$2-$4+100",", $8","$9-$4+$3+100"," }' >> nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq/nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq_genome.psl ;
[M::worker_pipeline::3764.259*0.18] mapped 2622 sequences
[M::main] Version: 2.17-r941
[M::main] CMD: /opt/JAFFA-version-2.3/tools/bin/minimap2 -t 4 -x splice -c /opt/JAFFA-version-2.3/hg38.fa nanopore2020_D10x_M95_benchmark_fusions.fasta.fastq/nanopore2020_D10x_M95_benchmark_fusions.fasta.fastq.fusions.fa
[M::main] Real time: 3777.054 sec; CPU: 680.451 sec; Peak RSS: 19.134 GB
Hello,
Thanks for trying out JAFFAL and posting these issues. These don't seem familiar to me, so would you mind addressing the questions detailed here, https://github.com/Oshlack/JAFFA/wiki/FAQandTroubleshooting#other-errors so I can get some more insight into what might be happening. In particular if you can email a reproducible example that would be a great help.
One of the error is about the bases and quality lengths being different for some reads. Did you check if this was indeed the case in your simulated data as that could be the cause of the failure?
Cheers, Nadia.
Thank you for getting back to me so quickly 😊 I ran JAFFAL on the simulated LongReadFusionSimulation Dataset and had some errors appear
======================== Stage extract_fusion_sequences (ONT_fus_sim_75err) ========================
bash: line 1: 32309 Killed /opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa Pac_fus_sim_80err.fastq/Pac_fus_sim_80err.fastq.fusions.fa > Pac_fus_sim_80err.fastq/Pac_fus_sim_80err.fastq_genome.paf
ERROR: Command failed with exit status = 137 :
/opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa Pac_fus_sim_80err.fastq/Pac_fus_sim_80err.fastq.fusions.fa > Pac_fus_sim_80err.fastq/Pac_fus_sim_80err.fastq_genome.paf; grep $'\t+\t' Pac_fus_sim_80err.fastq/Pac_fus_sim_80err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",",$3","$3+100",", $8","$9-$4+$3+100"," }' > Pac_fus_sim_80err.fastq/Pac_fus_sim_80err.fastq_genome.psl ; grep $'\t-\t' Pac_fus_sim_80err.fastq/Pac_fus_sim_80err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",", $2-$4","$2-$4+100",", $8","$9-$4+$3+100"," }' >> Pac_fus_sim_80err.fastq/Pac_fus_sim_80err.fastq_genome.psl ;
Done reading in transcript IDs
Reading the input alignment file, ONT_fus_sim_80err.fastq/ONT_fus_sim_80err.fastq.paf
0
15697 reads processed. Finished.
Done reading in transcript IDs
Reading the input alignment file, Pac_fus_sim_95err.fastq/Pac_fus_sim_95err.fastq.paf
============================ Stage minimap2_genome (Pac_fus_sim_95err) =============================
bash: line 1: 32315 Killed /opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa ONT_fus_sim_95err.fastq/ONT_fus_sim_95err.fastq.fusions.fa > ONT_fus_sim_95err.fastq/ONT_fus_sim_95err.fastq_genome.paf
[M::mm_idx_gen::629.794*0.24] collected minimizers
bash: line 1: 32396 Killed /opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa Pac_fus_sim_85err.fastq/Pac_fus_sim_85err.fastq.fusions.fa > Pac_fus_sim_85err.fastq/Pac_fus_sim_85err.fastq_genome.paf
ERROR: Command failed with exit status = 137 :
/opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa ONT_fus_sim_95err.fastq/ONT_fus_sim_95err.fastq.fusions.fa > ONT_fus_sim_95err.fastq/ONT_fus_sim_95err.fastq_genome.paf; grep $'\t+\t' ONT_fus_sim_95err.fastq/ONT_fus_sim_95err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",",$3","$3+100",", $8","$9-$4+$3+100"," }' > ONT_fus_sim_95err.fastq/ONT_fus_sim_95err.fastq_genome.psl ; grep $'\t-\t' ONT_fus_sim_95err.fastq/ONT_fus_sim_95err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",", $2-$4","$2-$4+100",", $8","$9-$4+$3+100"," }' >> ONT_fus_sim_95err.fastq/ONT_fus_sim_95err.fastq_genome.psl ;
ERROR: Command failed with exit status = 137 :
/opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa Pac_fus_sim_85err.fastq/Pac_fus_sim_85err.fastq.fusions.fa > Pac_fus_sim_85err.fastq/Pac_fus_sim_85err.fastq_genome.paf; grep $'\t+\t' Pac_fus_sim_85err.fastq/Pac_fus_sim_85err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",",$3","$3+100",", $8","$9-$4+$3+100"," }' > Pac_fus_sim_85err.fastq/Pac_fus_sim_85err.fastq_genome.psl ; grep $'\t-\t' Pac_fus_sim_85err.fastq/Pac_fus_sim_85err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",", $2-$4","$2-$4+100",", $8","$9-$4+$3+100"," }' >> Pac_fus_sim_85err.fastq/Pac_fus_sim_85err.fastq_genome.psl ;
bash: line 1: 32326 Killed /opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa Pac_fus_sim_90err.fastq/Pac_fus_sim_90err.fastq.fusions.fa > Pac_fus_sim_90err.fastq/Pac_fus_sim_90err.fastq_genome.paf
bash: line 1: 32633 Killed /opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa ONT_fus_sim_90err.fastq/ONT_fus_sim_90err.fastq.fusions.fa > ONT_fus_sim_90err.fastq/ONT_fus_sim_90err.fastq_genome.paf
[M::mm_idx_gen::944.330*0.18] collected minimizers
ERROR: Command failed with exit status = 137 :
/opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa Pac_fus_sim_90err.fastq/Pac_fus_sim_90err.fastq.fusions.fa > Pac_fus_sim_90err.fastq/Pac_fus_sim_90err.fastq_genome.paf; grep $'\t+\t' Pac_fus_sim_90err.fastq/Pac_fus_sim_90err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",",$3","$3+100",", $8","$9-$4+$3+100"," }' > Pac_fus_sim_90err.fastq/Pac_fus_sim_90err.fastq_genome.psl ; grep $'\t-\t' Pac_fus_sim_90err.fastq/Pac_fus_sim_90err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",", $2-$4","$2-$4+100",", $8","$9-$4+$3+100"," }' >> Pac_fus_sim_90err.fastq/Pac_fus_sim_90err.fastq_genome.psl ;
ERROR: Command failed with exit status = 137 :
/opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa ONT_fus_sim_90err.fastq/ONT_fus_sim_90err.fastq.fusions.fa > ONT_fus_sim_90err.fastq/ONT_fus_sim_90err.fastq_genome.paf; grep $'\t+\t' ONT_fus_sim_90err.fastq/ONT_fus_sim_90err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",",$3","$3+100",", $8","$9-$4+$3+100"," }' > ONT_fus_sim_90err.fastq/ONT_fus_sim_90err.fastq_genome.psl ; grep $'\t-\t' ONT_fus_sim_90err.fastq/ONT_fus_sim_90err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",", $2-$4","$2-$4+100",", $8","$9-$4+$3+100"," }' >> ONT_fus_sim_90err.fastq/ONT_fus_sim_90err.fastq_genome.psl ;
bash: line 1: 32332 Killed /opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa Pac_fus_sim_75err.fastq/Pac_fus_sim_75err.fastq.fusions.fa > Pac_fus_sim_75err.fastq/Pac_fus_sim_75err.fastq_genome.paf
ERROR: Command failed with exit status = 137 :
/opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa Pac_fus_sim_75err.fastq/Pac_fus_sim_75err.fastq.fusions.fa > Pac_fus_sim_75err.fastq/Pac_fus_sim_75err.fastq_genome.paf; grep $'\t+\t' Pac_fus_sim_75err.fastq/Pac_fus_sim_75err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",",$3","$3+100",", $8","$9-$4+$3+100"," }' > Pac_fus_sim_75err.fastq/Pac_fus_sim_75err.fastq_genome.psl ; grep $'\t-\t' Pac_fus_sim_75err.fastq/Pac_fus_sim_75err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",", $2-$4","$2-$4+100",", $8","$9-$4+$3+100"," }' >> Pac_fus_sim_75err.fastq/Pac_fus_sim_75err.fastq_genome.psl ;
[M::mm_idx_gen::417.599*0.33] collected minimizers
[M::mm_idx_gen::374.296*0.40] collected minimizers
[M::mm_idx_gen::2238.580*0.11] collected minimizers
bash: line 1: 32380 Killed /opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa ONT_fus_sim_85err.fastq/ONT_fus_sim_85err.fastq.fusions.fa > ONT_fus_sim_85err.fastq/ONT_fus_sim_85err.fastq_genome.paf
ERROR: Command failed with exit status = 137 :
/opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa ONT_fus_sim_85err.fastq/ONT_fus_sim_85err.fastq.fusions.fa > ONT_fus_sim_85err.fastq/ONT_fus_sim_85err.fastq_genome.paf; grep $'\t+\t' ONT_fus_sim_85err.fastq/ONT_fus_sim_85err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",",$3","$3+100",", $8","$9-$4+$3+100"," }' > ONT_fus_sim_85err.fastq/ONT_fus_sim_85err.fastq_genome.psl ; grep $'\t-\t' ONT_fus_sim_85err.fastq/ONT_fus_sim_85err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",", $2-$4","$2-$4+100",", $8","$9-$4+$3+100"," }' >> ONT_fus_sim_85err.fastq/ONT_fus_sim_85err.fastq_genome.psl ;
[M::mm_idx_gen::2702.171*0.11] collected minimizers
[M::mm_idx_gen::3319.178*0.12] sorted minimizers
[M::main::3319.178*0.12] loaded/built the index for 455 target sequence(s)
[M::mm_mapopt_update::3646.555*0.11] mid_occ = 792
[M::mm_idx_stat] kmer size: 15; skip: 5; is_hpc: 0; #seq: 455
[M::mm_idx_stat::3648.542*0.11] distinct minimizers: 167291034 (34.68% are singletons); average occurrences: 6.239; average spacing: 3.075
[M::worker_pipeline::5154.300*0.26] mapped 15855 sequences
[M::main] Version: 2.17-r941
[M::main] CMD: /opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa Pac_fus_sim_95err.fastq/Pac_fus_sim_95err.fastq.fusions.fa
[M::main] Real time: 5154.740 sec; CPU: 1343.433 sec; Peak RSS: 18.498 GB
========================================= Pipeline Failed ==========================================
One or more parallel stages aborted. The following messages were reported:
Branch ONT_fus_sim_85err.fastq in stage Unknown reported message:
Command failed with exit status = 137 :
/opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa ONT_fus_sim_85err.fastq/ONT_fus_sim_85err.fastq.fusions.fa > ONT_fus_sim_85err.fastq/ONT_fus_sim_85err.fastq_genome.paf; grep $'\t+\t' ONT_fus_sim_85err.fastq/ONT_fus_sim_85err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",",$3","$3+100",", $8","$9-$4+$3+100"," }' > ONT_fus_sim_85err.fastq/ONT_fus_sim_85err.fastq_genome.psl ; grep $'\t-\t' ONT_fus_sim_85err.fastq/ONT_fus_sim_85err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",", $2-$4","$2-$4+100",", $8","$9-$4+$3+100"," }' >> ONT_fus_sim_85err.fastq/ONT_fus_sim_85err.fastq_genome.psl ;
Branch ONT_fus_sim_90err.fastq in stage Unknown reported message:
Command failed with exit status = 137 :
/opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa ONT_fus_sim_90err.fastq/ONT_fus_sim_90err.fastq.fusions.fa > ONT_fus_sim_90err.fastq/ONT_fus_sim_90err.fastq_genome.paf; grep $'\t+\t' ONT_fus_sim_90err.fastq/ONT_fus_sim_90err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",",$3","$3+100",", $8","$9-$4+$3+100"," }' > ONT_fus_sim_90err.fastq/ONT_fus_sim_90err.fastq_genome.psl ; grep $'\t-\t' ONT_fus_sim_90err.fastq/ONT_fus_sim_90err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",", $2-$4","$2-$4+100",", $8","$9-$4+$3+100"," }' >> ONT_fus_sim_90err.fastq/ONT_fus_sim_90err.fastq_genome.psl ;
Branch ONT_fus_sim_95err.fastq in stage Unknown reported message:
Command failed with exit status = 137 :
/opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa ONT_fus_sim_95err.fastq/ONT_fus_sim_95err.fastq.fusions.fa > ONT_fus_sim_95err.fastq/ONT_fus_sim_95err.fastq_genome.paf; grep $'\t+\t' ONT_fus_sim_95err.fastq/ONT_fus_sim_95err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",",$3","$3+100",", $8","$9-$4+$3+100"," }' > ONT_fus_sim_95err.fastq/ONT_fus_sim_95err.fastq_genome.psl ; grep $'\t-\t' ONT_fus_sim_95err.fastq/ONT_fus_sim_95err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",", $2-$4","$2-$4+100",", $8","$9-$4+$3+100"," }' >> ONT_fus_sim_95err.fastq/ONT_fus_sim_95err.fastq_genome.psl ;
Branch Pac_fus_sim_75err.fastq in stage Unknown reported message:
Command failed with exit status = 137 :
/opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa Pac_fus_sim_75err.fastq/Pac_fus_sim_75err.fastq.fusions.fa > Pac_fus_sim_75err.fastq/Pac_fus_sim_75err.fastq_genome.paf; grep $'\t+\t' Pac_fus_sim_75err.fastq/Pac_fus_sim_75err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",",$3","$3+100",", $8","$9-$4+$3+100"," }' > Pac_fus_sim_75err.fastq/Pac_fus_sim_75err.fastq_genome.psl ; grep $'\t-\t' Pac_fus_sim_75err.fastq/Pac_fus_sim_75err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",", $2-$4","$2-$4+100",", $8","$9-$4+$3+100"," }' >> Pac_fus_sim_75err.fastq/Pac_fus_sim_75err.fastq_genome.psl ;
Branch Pac_fus_sim_80err.fastq in stage Unknown reported message:
Command failed with exit status = 137 :
/opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa Pac_fus_sim_80err.fastq/Pac_fus_sim_80err.fastq.fusions.fa > Pac_fus_sim_80err.fastq/Pac_fus_sim_80err.fastq_genome.paf; grep $'\t+\t' Pac_fus_sim_80err.fastq/Pac_fus_sim_80err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",",$3","$3+100",", $8","$9-$4+$3+100"," }' > Pac_fus_sim_80err.fastq/Pac_fus_sim_80err.fastq_genome.psl ; grep $'\t-\t' Pac_fus_sim_80err.fastq/Pac_fus_sim_80err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",", $2-$4","$2-$4+100",", $8","$9-$4+$3+100"," }' >> Pac_fus_sim_80err.fastq/Pac_fus_sim_80err.fastq_genome.psl ;
Branch Pac_fus_sim_85err.fastq in stage Unknown reported message:
Command failed with exit status = 137 :
/opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa Pac_fus_sim_85err.fastq/Pac_fus_sim_85err.fastq.fusions.fa > Pac_fus_sim_85err.fastq/Pac_fus_sim_85err.fastq_genome.paf; grep $'\t+\t' Pac_fus_sim_85err.fastq/Pac_fus_sim_85err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",",$3","$3+100",", $8","$9-$4+$3+100"," }' > Pac_fus_sim_85err.fastq/Pac_fus_sim_85err.fastq_genome.psl ; grep $'\t-\t' Pac_fus_sim_85err.fastq/Pac_fus_sim_85err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",", $2-$4","$2-$4+100",", $8","$9-$4+$3+100"," }' >> Pac_fus_sim_85err.fastq/Pac_fus_sim_85err.fastq_genome.psl ;
Branch Pac_fus_sim_90err.fastq in stage Unknown reported message:
Command failed with exit status = 137 :
/opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa Pac_fus_sim_90err.fastq/Pac_fus_sim_90err.fastq.fusions.fa > Pac_fus_sim_90err.fastq/Pac_fus_sim_90err.fastq_genome.paf; grep $'\t+\t' Pac_fus_sim_90err.fastq/Pac_fus_sim_90err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",",$3","$3+100",", $8","$9-$4+$3+100"," }' > Pac_fus_sim_90err.fastq/Pac_fus_sim_90err.fastq_genome.psl ; grep $'\t-\t' Pac_fus_sim_90err.fastq/Pac_fus_sim_90err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",", $2-$4","$2-$4+100",", $8","$9-$4+$3+100"," }' >> Pac_fus_sim_90err.fastq/Pac_fus_sim_90err.fastq_genome.psl ;
Use 'bpipe errors' to see output from failed commands.
ryleyd@minion:/data2/ryleyd/benchmark_simulated_reads/JAFFAL_troubleshooting$ /opt/JAFFA-version-2.3/tools/bin/bpipe errors
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.codehaus.groovy.reflection.CachedClass$3$1 (file:/opt/JAFFA-version-2.3/tools/bpipe-0.9.9.2/lib/bpipe.jar) to method java.lang.Object.finalize()
WARNING: Please consider reporting this to the maintainers of org.codehaus.groovy.reflection.CachedClass$3$1
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
============================== Found 7 failed commands from run 31354 ==============================
============================================ Command 64 ============================================
Command : /opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa Pac_fus_sim_80err.fastq/Pac_fus_sim_80err.fastq.fusions.fa > Pac_fus_sim_80err.fastq/Pac_fus_sim_80err.fastq_genome.paf; grep $'\t+\t' Pac_fus_sim_80err.fastq/Pac_fus_sim_80err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",",$3","$3+100",", $8","$9-$4+$3+100"," }' > Pac_fus_sim_80err.fastq/Pac_fus_sim_80err.fastq_genome.psl ; grep $'\t-\t' Pac_fus_sim_80err.fastq/Pac_fus_sim_80err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",", $2-$4","$2-$4+100",", $8","$9-$4+$3+100"," }' >> Pac_fus_sim_80err.fastq/Pac_fus_sim_80err.fastq_genome.psl ;
Exit Code : 137
Output :
No command output found in most recent log file
============================================ Command 56 ============================================
Command : /opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa ONT_fus_sim_95err.fastq/ONT_fus_sim_95err.fastq.fusions.fa > ONT_fus_sim_95err.fastq/ONT_fus_sim_95err.fastq_genome.paf; grep $'\t+\t' ONT_fus_sim_95err.fastq/ONT_fus_sim_95err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",",$3","$3+100",", $8","$9-$4+$3+100"," }' > ONT_fus_sim_95err.fastq/ONT_fus_sim_95err.fastq_genome.psl ; grep $'\t-\t' ONT_fus_sim_95err.fastq/ONT_fus_sim_95err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",", $2-$4","$2-$4+100",", $8","$9-$4+$3+100"," }' >> ONT_fus_sim_95err.fastq/ONT_fus_sim_95err.fastq_genome.psl ;
Exit Code : 137
Output :
No command output found in most recent log file
============================================ Command 62 ============================================
Command : /opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa Pac_fus_sim_90err.fastq/Pac_fus_sim_90err.fastq.fusions.fa > Pac_fus_sim_90err.fastq/Pac_fus_sim_90err.fastq_genome.paf; grep $'\t+\t' Pac_fus_sim_90err.fastq/Pac_fus_sim_90err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",",$3","$3+100",", $8","$9-$4+$3+100"," }' > Pac_fus_sim_90err.fastq/Pac_fus_sim_90err.fastq_genome.psl ; grep $'\t-\t' Pac_fus_sim_90err.fastq/Pac_fus_sim_90err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",", $2-$4","$2-$4+100",", $8","$9-$4+$3+100"," }' >> Pac_fus_sim_90err.fastq/Pac_fus_sim_90err.fastq_genome.psl ;
Exit Code : 137
Output :
No command output found in most recent log file
============================================ Command 60 ============================================
Command : /opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa Pac_fus_sim_75err.fastq/Pac_fus_sim_75err.fastq.fusions.fa > Pac_fus_sim_75err.fastq/Pac_fus_sim_75err.fastq_genome.paf; grep $'\t+\t' Pac_fus_sim_75err.fastq/Pac_fus_sim_75err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",",$3","$3+100",", $8","$9-$4+$3+100"," }' > Pac_fus_sim_75err.fastq/Pac_fus_sim_75err.fastq_genome.psl ; grep $'\t-\t' Pac_fus_sim_75err.fastq/Pac_fus_sim_75err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",", $2-$4","$2-$4+100",", $8","$9-$4+$3+100"," }' >> Pac_fus_sim_75err.fastq/Pac_fus_sim_75err.fastq_genome.psl ;
Exit Code : 137
Output :
No command output found in most recent log file
============================================ Command 66 ============================================
Command : /opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa ONT_fus_sim_85err.fastq/ONT_fus_sim_85err.fastq.fusions.fa > ONT_fus_sim_85err.fastq/ONT_fus_sim_85err.fastq_genome.paf; grep $'\t+\t' ONT_fus_sim_85err.fastq/ONT_fus_sim_85err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",",$3","$3+100",", $8","$9-$4+$3+100"," }' > ONT_fus_sim_85err.fastq/ONT_fus_sim_85err.fastq_genome.psl ; grep $'\t-\t' ONT_fus_sim_85err.fastq/ONT_fus_sim_85err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",", $2-$4","$2-$4+100",", $8","$9-$4+$3+100"," }' >> ONT_fus_sim_85err.fastq/ONT_fus_sim_85err.fastq_genome.psl ;
Exit Code : 137
Output :
No command output found in most recent log file
============================================ Command 74 ============================================
Command : /opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa Pac_fus_sim_85err.fastq/Pac_fus_sim_85err.fastq.fusions.fa > Pac_fus_sim_85err.fastq/Pac_fus_sim_85err.fastq_genome.paf; grep $'\t+\t' Pac_fus_sim_85err.fastq/Pac_fus_sim_85err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",",$3","$3+100",", $8","$9-$4+$3+100"," }' > Pac_fus_sim_85err.fastq/Pac_fus_sim_85err.fastq_genome.psl ; grep $'\t-\t' Pac_fus_sim_85err.fastq/Pac_fus_sim_85err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",", $2-$4","$2-$4+100",", $8","$9-$4+$3+100"," }' >> Pac_fus_sim_85err.fastq/Pac_fus_sim_85err.fastq_genome.psl ;
Exit Code : 137
Output :
No command output found in most recent log file
============================================ Command 86 ============================================
Command : /opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa ONT_fus_sim_90err.fastq/ONT_fus_sim_90err.fastq.fusions.fa > ONT_fus_sim_90err.fastq/ONT_fus_sim_90err.fastq_genome.paf; grep $'\t+\t' ONT_fus_sim_90err.fastq/ONT_fus_sim_90err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",",$3","$3+100",", $8","$9-$4+$3+100"," }' > ONT_fus_sim_90err.fastq/ONT_fus_sim_90err.fastq_genome.psl ; grep $'\t-\t' ONT_fus_sim_90err.fastq/ONT_fus_sim_90err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",", $2-$4","$2-$4+100",", $8","$9-$4+$3+100"," }' >> ONT_fus_sim_90err.fastq/ONT_fus_sim_90err.fastq_genome.psl ;
Exit Code : 137
Output :
No command output found in most recent log file
More information on directory contents
ryleyd@minion:~$ ls -lh /opt/JAFFA-version-2.3/
total 13G
-rwxrwxr-x 1 root root 2.3K Jul 24 2022 assemble.sh
-rwxrwxr-x 1 root root 2.5K Jul 24 2022 compile_results.R
-rw-rw-r-- 1 root root 35K Jul 24 2022 COPYING
drwxrwxr-x 2 root root 4.0K Jul 24 2022 cwl
drwxrwxr-x 2 root root 4.0K Jul 24 2022 docker
-rwxrwxr-x 1 root root 1.6K Jul 24 2022 get_spanning_reads.R
-rw-rw-r-- 1 1389338227 1389338227 3.1G Sep 11 2020 hg38.fa
-rwxrwxr-- 1 1389338227 1389338227 125M Jun 19 2015 hg38_genCode22.1.bt2
-rwxrwxr-- 1 1389338227 1389338227 71M Jun 19 2015 hg38_genCode22.2.bt2
-rwxrwxr-- 1 1389338227 1389338227 1.7M Jun 19 2015 hg38_genCode22.3.bt2
-rwxrwxr-- 1 1389338227 1389338227 71M Jun 19 2015 hg38_genCode22.4.bt2
-rw-r--r-- 1 1389338227 1389338227 36M Mar 5 2020 hg38_genCode22_blast.nhr
-rw-r--r-- 1 1389338227 1389338227 2.3M Mar 5 2020 hg38_genCode22_blast.nin
-rw-r--r-- 1 1389338227 1389338227 71M Mar 5 2020 hg38_genCode22_blast.nsq
-rwxrwxr-- 1 1389338227 1389338227 308M Jun 19 2015 hg38_genCode22.fa
-rwxrwxr-- 1 1389338227 1389338227 125M Jun 19 2015 hg38_genCode22.rev.1.bt2
-rwxrwxr-- 1 1389338227 1389338227 71M Jun 19 2015 hg38_genCode22.rev.2.bt2
-rwxrwxr-- 1 1389338227 1389338227 41M Jun 19 2015 hg38_genCode22.tab
-rwxrwxr-x 1 root root 6.5K Jul 24 2022 install_linux64.sh
-rwxrwxr-x 1 root root 1.2K Jul 24 2022 JAFFA_assembly.groovy
-rwxrwxr-x 1 root root 1.5K Jul 24 2022 JAFFA_direct.groovy
-rwxrwxr-x 1 root root 1.7K Jul 24 2022 JAFFA_hybrid.groovy
-rwxrwxr-x 1 root root 3.0K Jul 24 2022 JAFFAL.groovy
-rw-r--r-- 1 root root 4.9G Apr 25 2021 JAFFA_REFERENCE_FILES_HG38_GENCODE22.V2.tar.gz
-rwxrwxr-x 1 root root 21K Jul 24 2022 JAFFA_stages.groovy
-rw-r--r-- 1 root root 199K Jul 24 2022 JAFFA-version-2.3.tar.gz
-rw-rw-r-- 1 root root 400K Jul 24 2022 known_fusions.txt
-rw-rw-r-- 1 root root 818 Jul 24 2022 LICENSE
-rwxrwxr-x 1 root root 18K Jul 24 2022 make_final_table.R
-rwxrwxr-- 1 1389338227 1389338227 939M Mar 4 2015 Masked_hg38.1.bt2
-rwxrwxr-- 1 1389338227 1389338227 699M Mar 4 2015 Masked_hg38.2.bt2
-rwxrwxr-- 1 1389338227 1389338227 2.6M Mar 4 2015 Masked_hg38.3.bt2
-rwxrwxr-- 1 1389338227 1389338227 699M Mar 4 2015 Masked_hg38.4.bt2
-rwxrwxr-- 1 1389338227 1389338227 939M Mar 4 2015 Masked_hg38.rev.1.bt2
-rwxrwxr-- 1 1389338227 1389338227 699M Mar 4 2015 Masked_hg38.rev.2.bt2
-rwxrwxr-x 1 root root 618 Jul 24 2022 README
-rw-rw-r-- 1 root root 713 Jul 24 2022 README.md
drwxrwxr-x 2 root root 4.0K Jul 24 2022 scripts
drwxrwxr-x 2 root root 4.0K Jul 24 2022 src
drwxr-xr-x 13 root root 4.0K Feb 2 13:14 tools
-rw-r--r-- 1 root root 1017 Feb 2 13:15 tools.groovy
ryleyd@minion:~$ ls -lh /opt/
total 798M
drwxr-xr-x 9 root root 4.0K Feb 10 16:40 Aeron
-rw-r--r-- 1 root root 738M Oct 18 2022 Anaconda3-2022.10-Linux-x86_64.sh
drwxr-xr-x 9 root root 4.0K Oct 18 2022 Badread
drwxr-xr-x 3 root root 4.0K Feb 27 15:20 bsalign
drwx--x--x 4 root root 4.0K Jul 25 2022 containerd
drwxr-xr-x 4 root root 4.0K Jan 12 11:18 fusim-0.2.2
drwxr-xr-x 9 root root 4.0K Feb 3 09:42 genion
drwxr-xr-x 3 root root 4.0K Jan 16 14:47 google
drwxr-xr-x 3 rhysg users 4.0K Jun 20 14:35 gurobi1002
drwxrwxr-x 3 root root 4.0K Jun 20 14:12 ibm
drwxrwxr-x 7 root root 4.0K Feb 2 15:02 JAFFA-version-2.3
drwxr-xr-x 4 root root 4.0K Jun 8 17:24 nvidia
drwxr-xr-x 8 root root 4.0K Jun 20 16:11 ont
-rw-r--r-- 1 root root 61M Feb 24 2022 rstudio-server-2022.02.0-443-amd64.deb
-rw-r--r-- 1 root root 0 Mar 2 16:15 test_files02-03-2023test-meta.txt
drwxr-xr-x 12 root root 4.0K Jun 20 16:34 zoom
ryleyd@minion:~$ ls -lh /data2/ryleyd/benchmark_simulated_reads/positive_control/
total 259G
drwxr-xr-x 15 ryleyd users 4.0K Jun 19 16:06 nanopore
drwxr-xr-x 2 ryleyd users 4.0K Jun 6 13:02 original_fastq
drwxr-xr-x 15 ryleyd users 4.0K Jun 20 14:49 pacbio
-rw-r--r-- 1 ryleyd users 6.5G Jun 6 12:23 positive_control_nanopore2020_D100x_M85.fastq.gz
-rw-r--r-- 1 ryleyd users 18G Feb 24 22:44 positive_control_nanopore2020_D100x_M90.fastq.gz
-rw-r--r-- 1 ryleyd users 46G Feb 25 00:34 positive_control_nanopore2020_D100x_M95.fastq.gz
-rw-r--r-- 1 ryleyd users 3.3G May 29 15:40 positive_control_nanopore2020_D10x_M85.fastq.gz
-rw-r--r-- 1 ryleyd users 4.7G Feb 24 13:57 positive_control_nanopore2020_D10x_M90.fastq.gz
-rw-r--r-- 1 ryleyd users 4.6G Feb 24 14:26 positive_control_nanopore2020_D10x_M95.fastq.gz
-rw-r--r-- 1 ryleyd users 3.6G Feb 24 14:54 positive_control_nanopore2020_D20x_M85.fastq.gz
-rw-r--r-- 1 ryleyd users 9.3G Feb 24 15:17 positive_control_nanopore2020_D20x_M90.fastq.gz
-rw-r--r-- 1 ryleyd users 9.2G Feb 24 16:14 positive_control_nanopore2020_D20x_M95.fastq.gz
-rw-r--r-- 1 ryleyd users 4.8G Feb 24 17:10 positive_control_nanopore2020_D50x_M85.fastq.gz
-rw-r--r-- 1 ryleyd users 23G Feb 24 17:47 positive_control_nanopore2020_D50x_M90.fastq.gz
-rw-r--r-- 1 ryleyd users 23G Feb 24 19:57 positive_control_nanopore2020_D50x_M95.fastq.gz
-rw-r--r-- 1 ryleyd users 4.7G Feb 25 12:02 positive_control_pacbio2016_D100x_M85.fastq.gz
-rw-r--r-- 1 ryleyd users 11G Feb 25 12:39 positive_control_pacbio2016_D100x_M90.fastq.gz
-rw-r--r-- 1 ryleyd users 34G Feb 25 14:13 positive_control_pacbio2016_D100x_M95.fastq.gz
-rw-r--r-- 1 ryleyd users 2.5G Feb 25 04:43 positive_control_pacbio2016_D10x_M85.fastq.gz
-rw-r--r-- 1 ryleyd users 3.8G Feb 25 05:02 positive_control_pacbio2016_D10x_M90.fastq.gz
-rw-r--r-- 1 ryleyd users 3.7G Feb 25 05:31 positive_control_pacbio2016_D10x_M95.fastq.gz
-rw-r--r-- 1 ryleyd users 3.0G Feb 25 06:02 positive_control_pacbio2016_D20x_M85.fastq.gz
-rw-r--r-- 1 ryleyd users 4.7G Feb 25 06:24 positive_control_pacbio2016_D20x_M90.fastq.gz
-rw-r--r-- 1 ryleyd users 7.4G Feb 25 07:02 positive_control_pacbio2016_D20x_M95.fastq.gz
-rw-r--r-- 1 ryleyd users 3.7G Feb 25 08:02 positive_control_pacbio2016_D50x_M85.fastq.gz
-rw-r--r-- 1 ryleyd users 7.5G Feb 25 08:30 positive_control_pacbio2016_D50x_M90.fastq.gz
-rw-r--r-- 1 ryleyd users 19G Feb 25 09:33 positive_control_pacbio2016_D50x_M95.fastq.gz
-rw-r--r-- 1 ryleyd users 2.0K May 24 15:21 positive.fastfiles.md5sum.md5
drwxr-xr-x 2 ryleyd users 4.0K Jun 20 12:24 troubleshooting
ryleyd@minion:~$ cd /data2/ryleyd/benchmark_simulated_reads/positive_control/
ryleyd@minion:/data2/ryleyd/benchmark_simulated_reads/positive_control$ ls -lh /data2/ryleyd/benchmark_simulated_reads/positive_control/nanopore/
total 100K
-rw-r--r-- 1 ryleyd users 46 Jun 19 11:51 checks
-rw-r--r-- 1 ryleyd users 43K Jun 20 02:40 commandlog.txt
drwxr-xr-x 2 ryleyd users 4.0K Jun 19 15:57 positive_control_nanopore2020_D100x_M85.fastq
drwxr-xr-x 2 ryleyd users 4.0K Jun 19 11:56 positive_control_nanopore2020_D100x_M90.fastq
drwxr-xr-x 2 ryleyd users 4.0K Jun 20 02:40 positive_control_nanopore2020_D100x_M95.fastq
drwxr-xr-x 2 ryleyd users 4.0K Jun 19 15:57 positive_control_nanopore2020_D10x_M85.fastq
drwxr-xr-x 2 ryleyd users 4.0K Jun 19 16:08 positive_control_nanopore2020_D10x_M90.fastq
drwxr-xr-x 2 ryleyd users 4.0K Jun 19 16:06 positive_control_nanopore2020_D10x_M95.fastq
drwxr-xr-x 2 ryleyd users 4.0K Jun 19 13:07 positive_control_nanopore2020_D20x_M85.fastq
drwxr-xr-x 2 ryleyd users 4.0K Jun 19 20:15 positive_control_nanopore2020_D20x_M90.fastq
drwxr-xr-x 2 ryleyd users 4.0K Jun 19 20:15 positive_control_nanopore2020_D20x_M95.fastq
drwxr-xr-x 2 ryleyd users 4.0K Jun 19 16:06 positive_control_nanopore2020_D50x_M85.fastq
drwxr-xr-x 2 ryleyd users 4.0K Jun 19 16:13 positive_control_nanopore2020_D50x_M90.fastq
drwxr-xr-x 2 ryleyd users 4.0K Jun 19 23:33 positive_control_nanopore2020_D50x_M95.fastq
I'll respond later via email with a sample of my dataset that often has problems. I'll also make sure to check if the bases and quality lengths being different for some reads :) Thank you plenty
with regards to the error about the bases and quality lengths being different for some reads, this actually was the case. Some of the reads simulated by BadRead did not have a quality score line at all! 😮 So far JAFFAL has been the only tool to pick up on this.
Thanks for the update. Hopefully easy for you to fix the quality lengths in your simulation!
In regards to many of the others errors, it looks like it's failing at the minimap2 stage. Can test whether these commands work outside JAFFAL, for example by running: /opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa Pac_fus_sim_80err.fastq/Pac_fus_sim_80err.fastq.fusions.fa > Pac_fus_sim_80err.fastq/Pac_fus_sim_80err.fastq_genome.paf
and/or see if the file Pac_fus_sim_80err.fastq/Pac_fus_sim_80err.fastq_genome.paf etc. are non-empty.
Minimap2 can use a lot of RAM, in particular if you are running several jobs at once, so one thought it that you've hit the memory limit. How much memory does JAFFAL have access too? Do you see the same errors if you run one .fastq file at a time?
Cheers, Nadia.
@RDorney I've encountered the same Exit Code : 137 minimap2 issue when I tried running the tool on an AWS instance with 15 GB RAM. However, the error was fixed when running on an instance with more available RAM (I ran it successfully on an instance with 96 GB RAM). @nadiadavidson was right, minimap2 is pretty RAM intensive, running either one sample at a time or running it on a machine with more available RAM should fix this.
