vestigial -d option in pipeline causing error

[ablacutt]

running via singularity, the pipeline ends after hitting the following. looks as though this option was removed in an earlier commit - is this reflected in the current docker version? thanks!!

[Mon Nov 18 16:21:24 2019] Running: /usr/bin/perl /NGStools/metaerg/bin/output_reports.pl -d /data/db -g /data/my_metaerg_output/data/all.gff -o /data/my_metaerg_output -f /data/my_metaerg_output/metaerg.pl_11182019.fna
Unknown option: d
[Mon Nov 18 16:21:24 2019] Re-using existing --outdir /data/my_metaerg_output
DBI connect('dbname=/NGStools/metaerg/bin/../db/sqlite3/metaerg.db','',...) failed: unable to open database file at /NGStools/metaerg/bin/output_reports.pl line 58.
[Mon Nov 18 16:21:24 2019] Could not run command:/usr/bin/perl /NGStools/metaerg/bin/output_reports.pl -d /data/db -g /data/my_metaerg_output/data/all.gff -o /data/my_metaerg_output -f /data/my_metaerg_output/metaerg.pl_11182019.fna,

[shafferm]

I am running into this same issue when installed locally.

[CaitlinCasar]

I'm running into this issue installing on a remote server using both the instructions in this repo as well as with MetaErgInstallHelper

[Finesim97]

I don't know why the option was removed. For now here is a duct tape solution for local installations:

# metaerg is the dir with the bin folder
mkdir -p metaerg/db/sqlite3
ln -s metaergdata/sqlite3/metaerg.db  metaerg/db/sqlite3/metaerg.db

While this allows the reporting to start, it crashes on this line, when no depth data is available: https://github.com/xiaoli-dong/metaerg/blob/532bd189a446f8da85373335d48105ace49f3f7e/bin/output_reports.pl#L1027

This crash and the database location have been fixed in previous commits. The changes in those commits were however reverted with the latest one. I am not sure which changes belong to the rRNA fix and which aren't supposed to be there.

Local Fix

For now you can use the version before the lastest commit by switching into the metaerg installation directory (metaerginstall/metaerg if you used my helper) and run:

git checkout 68907b737b12c7456275b028ea31314aebbe50e

Docker Fix

To get a working docker image run: https://gist.github.com/Finesim97/6b7635d526bba47f8cbb9f609440d688

[CaitlinCasar]

the local fix worked for me, thank you!

[MeloOspino]

Hi. I had the same problem after I analysed my samples using the docker version. And I was not able to make the docker fix image to work. Some dependencies were not installed after I run docker build -t metaerg-old .

[Finesim97]

Ok, weird. I remember it working, will try it tomorrow. Can you post the error message?

[MeloOspino]

Hi @Finesim97 Thanks you for your quick answer. Some of the error that appear after I run the code docker build -t metaerg-old . :

==> Found dependencies: Bio::DB::WebDBSeqI --> Working on Bio::DB::EMBL Fetching http://www.cpan.org/authors/id/C/CJ/CJFIELDS/Bio-DB-EMBL-1.7.4.tar.gz ... OK Configuring Bio-DB-EMBL-1.7.4 ... ! Installing the dependencies failed: Module 'Bio::DB::DBFetch' is not installed ! Bailing out the installation for Bio-DB-EMBL-1.7.4. ! Installing the dependencies failed: Module 'Bio::DB::GenPept' is not installed, Module 'Bio::Tools::Run::RemoteBlast' is not installed, Module 'Bio::DB::SwissProt' is not installed, Module 'Bio::DB::EMBL' is not installed, Module 'Bio::DB::RefSeq' is not installed, Module 'Bio::Root::Version' is not installed, Module 'Bio::DB::GenBank' is not installed, Module 'Bio::Seq' is not installed, Module 'Bio::SeqIO' is not installed, Module 'Bio::Root::Test' is not installed ! Bailing out the installation for Bio-Procedural-1.7.4. Configuring XML-Simple-2.25 ... OK ==> Found dependencies: XML::SAX::Expat --> Working on XML::SAX::Expat Fetching http://www.cpan.org/authors/id/B/BJ/BJOERN/XML-SAX-Expat-0.51.tar.gz ... OK Configuring XML-SAX-Expat-0.51 ... OK Building and testing XML-SAX-Expat-0.51 ... OK Successfully installed XML-SAX-Expat-0.51 Building and testing XML-Simple-2.25 ... OK Successfully installed XML-Simple-2.25 ! Installing the dependencies failed: Module 'Bio::DB::GenericWebAgent' is not installed, Module 'Bio::ParameterBaseI' is not installed, Module 'Bio::Root::RootI' is not installed, Module 'Bio::SeqIO' is not installed, Module 'Bio::Root::Root' is not installed, Module 'Bio::ASN1::EntrezGene' is not installed, Module 'Bio::Root::Version' is not installed, Module 'Bio::Root::IO' is not installed ! Bailing out the installation for Bio-EUtilities-1.75. 83 distributions installed The command '/bin/sh -c cpanm Bio::Perl DBI Archive::Extract DBD::SQLite File::Copy::Recursive Bio::DB::EUtilities LWP::Protocol::https && git clone https://git.code.sf.net/p/swissknife/git swissknife-git && cd swissknife-git && perl Makefile.PL && make install && cd /NGStools' returned a non-zero code: 1

And if I check for the dockers list, this is what appears:

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES f216cbf51552 63d857cebf82 "/bin/sh -c 'cpanm B…" 20 minutes ago Exited (1) 4 minutes ago cocky_hamilton 92b69a759bb0 63d857cebf82 "/bin/sh -c 'cpanm B…" 37 hours ago Exited (1) 37 hours ago dazzling_liskov 7211af74a8a5 63d857cebf82 "/bin/sh -c 'cpanm B…" 43 hours ago Exited (1) 43 hours ago optimistic_jackson

Thank you for the help.

[Finesim97]

Ok, noticed two problems with the Dockerfile. The module XML-DOM-XPath used by BioPerl still uses the old encoding pragma in one test, which leads to a crash with a current Perl version. This can be circumvented by telling cpanminus to ignore test results (-f). Furthermore BLAST+ 2.10 was released in December, breaking the 2.9 download link in the Dockerfile. 2.10 uses the version 5 database format per default in makeblastdb, don't know if it is okay to update. Should be no problem, but with my bad luck lately, for now I would continue to use the 2.9 version.

To create the image I wrote a patch for the Dockerfile for you: https://gist.github.com/Finesim97/6b7635d526bba47f8cbb9f609440d688

[Finesim97]

Wrote a PR #9 for the problems with the Dockerfile, that installs the current version. This fixes the problems with the Dockerfile, but not the missing option and depth data crash.

[MeloOspino]

Hi @Finesim97 It worked for the test files. =D Now I will use my actual data. Finger cruz.
Thank you so much for your help!

[PatoUru]

Hi @Finesim97! I hope that you are very well!! I tried the first option: (base) patricia@nzxt:/media/patricia/f21f0a48-f5fe-411d-bad9-a8c6b2348fe5/Programas/MetaErg$ mkdir -p metaerg/db/sqlite3 (base) patricia@nzxt:/media/patricia/f21f0a48-f5fe-411d-bad9-a8c6b2348fe5/Programas/MetaErg$ ln -s metaergdata/sqlite3/metaerg.db metaerg/db/sqlite3/metaerg.db

But it does not work :(

The error persist

$ /media/patricia/f21f0a48-f5fe-411d-bad9-a8c6b2348fe5/Programas/MetaErg/metaerg/bin/output_reports.pl -g output-RH_29/tmp/features.gff -f RH_29.fna -o output_reports [Fri Jan 24 17:59:19 2020] Re-using existing --outdir output_reports DBI connect('dbname=/media/patricia/f21f0a48-f5fe-411d-bad9-a8c6b2348fe5/Programas/MetaErg/metaerg/bin/../db/sqlite3/metaerg.db','',...) failed: unable to open database file at /media/patricia/f21f0a48-f5fe-411d-bad9-a8c6b2348fe5/Programas/MetaErg/metaerg/bin/output_reports.pl line 58.

The initial script was $ perl /media/patricia/f21f0a48-f5fe-411d-bad9-a8c6b2348fe5/Programas/MetaErg/metaerg/bin/metaerg.pl --dbdir /media/patricia/f21f0a48-f5fe-411d-bad9-a8c6b2348fe5/database/db_MetaEG/db --outdir out-RH_29 --prefix RH_29 --cpus 12 RH_29_FINAL.fa

How could I fix it? Thank you in advance!! Patricia

[pvdam3]

I am still experiencing problems using the Docker instance, but managed to get the output_reports.pl script to run by binding the db folder to the location where the script by default searches inside the Docker (/NGStools/metaerg/db). Maybe this proves useful for someone else:

docker run --shm-size 2g --rm -u $(id -u):$(id -g) -it -v /media:/media -v /home:/home -v /media/Fileserver12/Groups/BioInf/MetaErg/db:/NGStools/metaerg/db xiaolidong/docker-metaerg /NGStools/metaerg/bin/output_reports.pl -g '/home/peter/Desktop/out5/data/all.gff' -o /home/peter/Desktop/out5 -f /home/peter/Desktop/out5/metaerg.pl_12222020.fna