Error 1 when running nanocorrect-overlap.make

[BoMatt]

Hi! When i run nanocorrect-overlap.make i get this error:

make -f nanocorrect-overlap.make INPUT=~/MinION/sspace_butyricum/all_reads_buty.fasta NAME=cor

nanocorrect-preprocess.pl /home/mattia/MinION/sspace_butyricum/all_reads_buty.fasta > cor.pp.fasta fasta2DB cor cor.pp.fasta File cor.pp.fasta, Line 1502272: Warning: Read is longer than 2^16-1. Read truncated. File cor.pp.fasta, Line 1537720: Warning: Read is longer than 2^16-1. Read truncated. File cor.pp.fasta, Line 1845517: Warning: Read is longer than 2^16-1. Read truncated. File cor.pp.fasta, Line 1998480: Warning: Read is longer than 2^16-1. Read truncated. File cor.pp.fasta, Line 2004327: Warning: Read is longer than 2^16-1. Read truncated. File cor.pp.fasta, Line 2270391: Warning: Read is longer than 2^16-1. Read truncated. File cor.pp.fasta, Line 2357344: Warning: Read is longer than 2^16-1. Read truncated. File cor.pp.fasta, Line 2433997: Warning: Read is longer than 2^16-1. Read truncated. File cor.pp.fasta, Line 2648752: Warning: Read is longer than 2^16-1. Read truncated. File cor.pp.fasta, Line 2998765: Warning: Read is longer than 2^16-1. Read truncated. File cor.pp.fasta, Line 3020306: Warning: Read is longer than 2^16-1. Read truncated. DBsplit -s50 cor DBdust cor HPCdaligner -t5 -mdust cor > HPCcommands.txt /bin/bash HPCcommands.txt HPCcommands.txt: line 2: 22420 Segmentation fault (core dumped) daligner -t5 -mdust cor.1 cor.1 HPCcommands.txt: line 3: 22422 Segmentation fault (core dumped) daligner -t5 -mdust cor.2 cor.1 cor.2 daligner: Block cor.3 contains reads < 14bp long ! Run DBsplit. daligner: Block cor.4 contains reads < 14bp long ! Run DBsplit. LAsort: Cannot open ./cor.1.cor.1.C0.las for 'r' LAsort: Cannot open ./cor.1.cor.2.C0.las for 'r' LAsort: Cannot open ./cor.1.cor.3.C0.las for 'r' LAsort: Cannot open ./cor.1.cor.4.C0.las for 'r' LAsort: Cannot open ./cor.2.cor.1.C0.las for 'r' LAsort: Cannot open ./cor.2.cor.2.C0.las for 'r' LAsort: Cannot open ./cor.2.cor.3.C0.las for 'r' LAsort: Cannot open ./cor.2.cor.4.C0.las for 'r' LAsort: Cannot open ./cor.3.cor.1.C0.las for 'r' LAsort: Cannot open ./cor.3.cor.2.C0.las for 'r' LAsort: Cannot open ./cor.3.cor.3.C0.las for 'r' LAsort: Cannot open ./cor.3.cor.4.C0.las for 'r' LAsort: Cannot open ./cor.4.cor.1.C0.las for 'r' LAsort: Cannot open ./cor.4.cor.2.C0.las for 'r' LAsort: Cannot open ./cor.4.cor.3.C0.las for 'r' LAsort: Cannot open ./cor.4.cor.4.C0.las for 'r' LAmerge: Cannot open ./L1.1.1.las for 'r' LAmerge: Cannot open ./L1.2.1.las for 'r' LAmerge: Cannot open ./L1.3.1.las for 'r' LAmerge: Cannot open ./L1.4.1.las for 'r' nanocorrect-overlap.make:37: recipe for target 'cor.las' failed make: *** [cor.las] Error 1

All dependencies are updated to the latest version, which can be the problem?

[jts]

The warnings about reads that are too long and too short (<14bp) might indicate the cause. Perhaps you can pre-filter your data to get rid of this and see if it works?

Is your input data 2D nanopore reads?

[BoMatt]

Yes,the input is 2D nanopore reads. I will try to filter out the short reads and see if it works.

[BoMatt]

In the FASTA there weren't reads shorter than 100bp, I tried to filter out also reads shorter than 500bp with two different tools but I'm getting the same error.

make -f nanocorrect-overlap.make INPUT=~/MinION/sspace_butyricum/all2Dbutyricum_filter_galaxy.fasta NAME=gala

nanocorrect-preprocess.pl /home/mattia/MinION/sspace_butyricum/all2Dbutyricum_filter_galaxy.fasta > gala.pp.fasta fasta2DB gala gala.pp.fasta DBsplit -s50 gala DBdust gala HPCdaligner -t5 -mdust gala > HPCcommands.txt /bin/bash HPCcommands.txt daligner: Block gala.1 contains reads < 14bp long ! Run DBsplit. LAsort: Cannot open ./gala.1.gala.1.C0.las for 'r' nanocorrect-overlap.make:37: recipe for target 'gala.las' failed make: *** [gala.las] Error 1

[jts]

Can you make the input file available to me so I can look into this?

[BoMatt]

I don't know if it works in Google Drive, let me know. https://drive.google.com/file/d/0BzIk2g1Es23ZMURRdjFWUm9MOVE/view

[jts]

I just tested your file and it worked for me:

>make -f nanocorrect/nanocorrect-overlap.make INPUT=all2Dbutyricum.fasta NAME=test.v1
nanocorrect-preprocess.pl all2Dbutyricum.fasta > test.v1.pp.fasta
fasta2DB test.v1 test.v1.pp.fasta
DBsplit -s50 test.v1
DBdust test.v1
HPCdaligner -t5 -mdust test.v1 > HPCcommands.txt
/bin/bash HPCcommands.txt
LAcat test.v1 > test.v1.las
rm test.v1.*.las

Can you try it again in a completely clean directory? daligner uses hidden files and if these aren't removed it can make subsequent runs crash.

[BoMatt]

I tried it again but always the same error...I don't know how to solve it if I have to be honest, it seems a problem of my PC or of DALIGNER. Also because anytime I run it, then I delete the hidden files.

[jts]

Ok. What OS are you running and what versions of python/daligner?

[BoMatt]

Ubuntu 15.04, Python 2.7.9, daligner I think it's the last version because the git pull says that is already up to date.

[jts]

Can you run the exact command that I pasted above and send me the md5sums of all files?

[BoMatt]

Let me know if you need something else.

3b0c5878414f197d631e176ff90195f9 test.v1.db 6df6413e3fd993c5e89821698a66a16f test.v1.pp.fasta d54bd5b69d039b84f7420c9495d26ea4 HPCcommands.txt 3cfa46cfb891db70145ae6cbc246ba53 .test.v1.bps b921d466c5e407f8a5bf5e5ebd14f94c .test.v1.dust.anno 98a015b601fd277736877f622b9c089c .test.v1.dust.data 34fbe33e98d072c4b0cbfce5d09ecfa9 .test.v1.idx

[jts]

They match my files except for .test.v1.dust.anno, .test.v1.dust.data and .test.v1.idx. I don't know why running DBdust on the same file (test.v1.pp.fasta) would give different results.

[joshquick]

I’ve noticed that before whilst running the same input files on two different VM’s

On 18 Aug 2015, at 15:50, Jared Simpson <[email protected]mailto:[email protected]> wrote:

They match my files except for .test.v1.dust.anno, .test.v1.dust.data and .test.v1.idx. I don't know why running DBdust on the same file (test.v1.pp.fasta) would give different results.

— Reply to this email directly or view it on GitHubhttps://github.com/jts/nanocorrect/issues/14#issuecomment-132237979.

[BoMatt]

Tried to delete and reinstall DAZZ_DB, now it seems working!

[jts]

Great!

[BoMatt]

And now I have again the same error! It's quite strange...

[BoMatt]

I tried with different parameters, it seems that you have to create every time a new folder and delete to old one, or at least in this way it's working for me.

[jts]

Have you tried running DAZZ_DB/DBrm in between runs? Its the proper way of cleaning up the hidden files, which I think might be the problem

[BoMatt]

I will let you know when I'll do the next run, didn't know there was that command to remove the files. Thanks for the help!

[seppinho]

Hi, I'm currently experiencing the same issues as reported by MattiaBo with my 2D reads. Output /bin/bash HPCcommands.txt HPCcommands.txt: line 2: 19 Segmentation fault (core dumped) daligner -t5 -mdust test2.1 test2.1 LAsort: Cannot open ./test2.1.test2.1.C0.las for 'r'

I did everything within Docker and started with a fresh Ubuntu 14.04 image with python 2.7.6. So I don't think it's related to the database. Things I've installed:

• Latest daligner + DAZZ_DB (git clone)
• poaV2
• bwa (git clone; not sure if it's required but its included in your env.sh so I installed it)
• bioperl (installed with apt-get)

I could also provide my fasta file if necessary. Thanks for your help! Sebastian

[jts]

I've just noticed that daligner and dazz_db were recently updated. Would you mind trying the versions that we used in our earlier paper to see if the old code works on your data? The commit IDs are here:

https://github.com/jts/nanopore-paper-analysis/blob/master/full-pipeline.make#L83

[seppinho]

Hej, indeed, that did the trick. Thanks a lot for the help!! DALIGNER version: git checkout 549da77b91395dd && make DAZZ_DB version: git checkout 8cb2f29c4011a2c2 && make

[jts]

Thanks for testing that. It looks like I'll have to modify nanocorrect to be compatible with the latest daligner

[seppinho]

happy to help

[EvdH0]

I can second this, with the current DALIGNER and DAZZ_DB version I get a segmentation fault

make -f nanocorrect-overlap.make INPUT=2D.min800bp.fasta NAME=nc /bin/bash HPCcommands.txt HPCcommands.txt: line 2: 11942 Segmentation fault: 11 daligner -t5 -mdust nc.1 nc.1 LAsort: Cannot open ./nc.1.nc.1.C0.las for 'r' make: *** [nc.las] Error 1

however going back to the two versions @seppinho mentioned and are in the makefile (https://github.com/jts/nanopore-paper-analysis/blob/master/full-pipeline.make#L83) it indeed works.

[duspriya]

Hi, I'm currently experiencing the same issues as reported

make -f nanocorrect-overlap.make INPUT=reads.fasta NAME=test.v1 nanocorrect-preprocess.pl reads.fasta > test.v1.pp.fasta /bin/bash: nanocorrect-preprocess.pl: command not found nanocorrect-overlap.make:14: recipe for target 'test.v1.pp.fasta' failed make: *** [test.v1.pp.fasta] Error 127 All dependencies are updated to the latest version, which can be the problem?

[jts]

Hi @duspriya,

As noted in the README, this software is deprecated and should not be used. I suggest using racon for error correction instead.

In case you are curious though the problem is that the nanocorrect-preprocess.pl script could not be found. You should check whether it is present in the same directory as nanocorrect-overlap.make.

Jared