DRAM
DRAM copied to clipboard
DRAM-v.py annotate
-
We are able to launch this command and get all the database set up and files. DRAM-setup.py prepare_databases --output_dir dramv --skip_uniref
-
This is the content of the configuration file: { "search_databases": { "kegg": null, "kofam_hmm": "/scratch/lauhotte/dramv/kofam_profiles.hmm", "kofam_ko_list": "/scratch/lauhotte/dramv/kofam_ko_list.tsv", "uniref": null, "pfam": "/scratch/lauhotte/dramv/pfam.mmspro", "dbcan": "/scratch/lauhotte/dramv/dbCAN-HMMdb-V11.txt", "viral": "/scratch/lauhotte/dramv/refseq_viral.20231218.mmsdb", "peptidase": "/scratch/lauhotte/dramv/peptidases.20231218.mmsdb", "vogdb": "/scratch/lauhotte/dramv/vog_latest_hmms.txt" }, "custom_dbs": null, "database_descriptions": { "pfam_hmm_dat": "/scratch/lauhotte/dramv/Pfam-A.hmm.dat.gz", "dbcan_fam_activities": "/scratch/lauhotte/dramv/CAZyDB.08062022.fam-activities.txt", "vog_annotations": "/scratch/lauhotte/dramv/vog_annotations_latest.tsv.gz" }, "dram_sheets": { "genome_summary_form": "/scratch/lauhotte/dramv/genome_summary_form.20231218.tsv", "module_step_form": "/scratch/lauhotte/dramv/module_step_form.20231218.tsv", "etc_module_database": "/scratch/lauhotte/dramv/etc_mdoule_database.20231218.tsv", "function_heatmap_form": "/scratch/lauhotte/dramv/function_heatmap_form.20231218.tsv", "amg_database": "/scratch/lauhotte/dramv/amg_database.20231218.tsv" }, "description_db": "/scratch/lauhotte/dramv/description_db.sqlite", "dram_version": "1.4.6" }
-
Here is the command to launch DRAM-v.py annotate in an interactive session where dram module is loaded: DRAM-v.py annotate -v ~/scratch/pass2_fordramv/WGA4S5/for-dramv/viral-affi-contigs-for-dramv.tab -i ~/scratch/pass2_fordramv/WGA4S5/for-dramv/final-viral-combined-for-dramv.fa -o WGA4S5-HG-test-22 --skip_trnascan --threads 2 --min_contig_size 1000
-
Here is the output error message: ombined-for-dramv.fa -o WGA4S5-HG-test-21 --skip_trnascan --threads 2 --min_contig_size 1000 2023-12-21 12:11:30,002 - 1 fastas found 2023-12-21 12:11:30,890 - Checking for duplicate names 2023-12-21 12:11:30,938 - No duplicate names found 2023-12-21 12:11:30,938 - Starting the Viral Annotation of Genes with database configuration: there are no settings, the config is corrupted or too old. 2023-12-21 12:11:30,939 - Retrieved database locations and descriptions 2023-12-21 12:11:30,939 - Annotating final-viral-combined-for-dramv 2023-12-21 12:11:32,411 - Turning genes from prodigal to mmseqs2 db 2023-12-21 12:11:48,003 - Getting hits from kofam 2023-12-21 12:12:41,299 - Getting forward best hits from viral 2023-12-21 12:13:00,616 - Getting reverse best hits from viral 2023-12-21 12:13:08,194 - Getting descriptions of hits from viral 2023-12-21 12:13:12,459 - Getting forward best hits from peptidase 2023-12-21 12:13:25,473 - Getting reverse best hits from peptidase 2023-12-21 12:13:28,175 - Getting descriptions of hits from peptidase 2023-12-21 12:13:29,433 - Getting hits from pfam 2023-12-21 12:21:42,781 - Getting hits from dbCAN 2023-12-21 12:21:49,531 - Getting hits from VOGDB 2023-12-21 12:22:21,719 - Merging ORF annotations Traceback (most recent call last): File "/home/lauhotte/ENV_dram/bin/DRAM-v.py", line 161, in
args.func(**args_dict) File "/home/lauhotte/ENV_dram/lib/python3.10/site-packages/mag_annotator/annotate_vgfs.py", line 484, in annotate_vgfs annotations = annotate_fastas( File "/home/lauhotte/ENV_dram/lib/python3.10/site-packages/mag_annotator/annotate_bins.py", line 1394, in annotate_fastas annotate_fasta( File "/home/lauhotte/ENV_dram/lib/python3.10/site-packages/mag_annotator/annotate_bins.py", line 1306, in annotate_fasta rrna_table = run_barrnap( File "/home/lauhotte/ENV_dram/lib/python3.10/site-packages/mag_annotator/annotate_bins.py", line 684, in run_barrnap raw_rrna_str = run_process( File "/home/lauhotte/ENV_dram/lib/python3.10/site-packages/mag_annotator/utils.py", line 61, in run_process results = subprocess.run(command, check=check, shell=shell, File "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Core/python/3.10.2/lib/python3.10/subprocess.py", line 501, in run with Popen(*popenargs, **kwargs) as process: File "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Core/python/3.10.2/lib/python3.10/subprocess.py", line 966, in init self._execute_child(args, executable, preexec_fn, close_fds, File "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Core/python/3.10.2/lib/python3.10/subprocess.py", line 1842, in _execute_child raise child_exception_type(errno_num, err_msg, err_filename) FileNotFoundError: [Errno 2] No such file or directory: 'barrnap'
- Here is the command we launch in a compute node: #!/bin/bash #SBATCH --account=def-smoineau #SBATCH --mem=25G #SBATCH --cpus-per-task=4 #SBATCH --time=1:00:00
module load gcc/9.3.0 python/3.10 hmmer/3.2.1 barrnap/0.9 prodigal/2.6.3 mmseqs2/14-7e284 trnascan-se/2.0.12
virtualenv --no-download $SLURM_TMPDIR/env source $SLURM_TMPDIR/env/bin/activate pip install --no-index --upgrade pip pip install --no-index -r requirements_dramv.txt
cd ~/scratch/dramv/
DRAM-v.py annotate -v ~/scratch/pass2_fordramv/WGA4S5/for-dramv/viral-affi-contigs-for-dramv.tab -i ~/scratch/pass2_fordramv/WGA4S5/for-dramv/final-viral-combined-for-dramv.fa -o WGA4S5-HG-test-22 --skip_trnascan --threads $SLURM_CPUS_PER_TASK --min_contig_size 1000
- Here we get another error messages: Lmod is automatically replacing "intel/2020.1.217" with "gcc/9.3.0".
Due to MODULEPATH changes, the following have been reloaded:
- openmpi/4.0.3
created virtual environment CPython3.10.2.final.0-64 in 4781ms
creator CPython3Posix(dest=/localscratch/lauhotte.20827345.0/env, clear=False, no_vcs_ignore=False, global=False)
seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/lauhotte/.local/share/virtualenv)
added seed packages: pip==23.3.2, setuptools==69.0.2, wheel==0.41.3
activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator
Looking in links: /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/gentoo2020/avx2, /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/gentoo/avx2, /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/gentoo2020/generic, /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/gentoo/generic, /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/generic
Requirement already satisfied: pip in /localscratch/lauhotte.20827345.0/env/lib/python3.10/site-packages (23.3.2)
Looking in links: /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/gentoo2020/avx2, /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/gentoo/avx2, /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/gentoo2020/generic, /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/gentoo/generic, /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/generic
Processing /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/generic/altair-5.1.2+computecanada-py3-none-any.whl (from -r requirements_dramv.txt (line 1))
Processing /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/generic/asttokens-2.4.1+computecanada-py2.py3-none-any.whl (from -r requirements_dramv.txt (line 2))
Processing /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/generic/attrs-23.1.0+computecanada-py3-none-any.whl (from -r requirements_dramv.txt (line 3))
Processing /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/generic/certifi-2023.7.22+computecanada-py3-none-any.whl (from -r requirements_dramv.txt (line 4))
Processing /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/generic/charset_normalizer-3.2.0+computecanada-py3-none-any.whl (from -r requirements_dramv.txt (line 5))
Processing /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/generic/contourpy-1.1.1+computecanada-cp310-cp310-linux_x86_64.whl (from -r requirements_dramv.txt (line 6))
Processing /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/generic/cycler-0.12.1+computecanada-py3-none-any.whl (from -r requirements_dramv.txt (line 7))
Processing /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/generic/Cython-3.0.4+computecanada-cp310-cp310-linux_x86_64.whl (from -r requirements_dramv.txt (line 8))
Processing /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/generic/decorator-5.1.1+computecanada-py3-none-any.whl (from -r requirements_dramv.txt (line 9))
Processing /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/generic/DRAM_bio-1.4.6+computecanada-py3-none-any.whl (from -r requirements_dramv.txt (line 10))
Processing /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/generic/et_xmlfile-1.1.0+computecanada-py3-none-any.whl (from -r requirements_dramv.txt (line 11))
Processing /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/generic/exceptiongroup-1.1.3+computecanada-py3-none-any.whl (from -r requirements_dramv.txt (line 12))
Processing /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/generic/executing-2.0.1+computecanada-py2.py3-none-any.whl (from -r requirements_dramv.txt (line 13))
Processing /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/generic/fonttools-4.46.0+computecanada-cp310-cp310-linux_x86_64.whl (from -r requirements_dramv.txt (line 14))
Processing /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/generic/greenlet-2.0.2+computecanada-cp310-cp310-linux_x86_64.whl (from -r requirements_dramv.txt (line 15))
Processing /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/gentoo/avx2/h5py-3.8.0+computecanada-cp310-cp310-linux_x86_64.whl (from -r requirements_dramv.txt (line 16))
Processing /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/generic/hdmedians-0.14.2+computecanada-cp310-cp310-linux_x86_64.whl (from -r requirements_dramv.txt (line 17))
Processing /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/generic/idna-3.4+computecanada-py3-none-any.whl (from -r requirements_dramv.txt (line 18))
Processing /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/generic/ipython-8.17.2+computecanada-py3-none-any.whl (from -r requirements_dramv.txt (line 19))
Processing /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/generic/jedi-0.19.1+computecanada-py2.py3-none-any.whl (from -r requirements_dramv.txt (line 20))
Processing /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/generic/Jinja2-3.1.2+computecanada-py3-none-any.whl (from -r requirements_dramv.txt (line 21))
Processing /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/generic/jsonschema-4.20.0+computecanada-py3-none-any.whl (from -r requirements_dramv.txt (line 22))
Processing /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/generic/jsonschema_specifications-2023.11.2+computecanada-py3-none-any.whl (from -r requirements_dramv.txt (line 23))
Processing /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/gentoo/avx2/kiwisolver-1.4.5+computecanada-cp310-cp310-linux_x86_64.whl (from -r requirements_dramv.txt (line 24))
Processing /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/generic/MarkupSafe-2.1.3+computecanada-cp310-cp310-linux_x86_64.whl (from -r requirements_dramv.txt (line 25))
Processing /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/gentoo/avx2/matplotlib-3.7.2+computecanada-cp310-cp310-linux_x86_64.whl (from -r requirements_dramv.txt (line 26))
Processing /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/generic/matplotlib_inline-0.1.6+computecanada-py3-none-any.whl (from -r requirements_dramv.txt (line 27))
Processing /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/generic/natsort-8.4.0+computecanada-py3-none-any.whl (from -r requirements_dramv.txt (line 28))
Processing /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/generic/networkx-3.2.1+computecanada-py3-none-any.whl (from -r requirements_dramv.txt (line 29))
Processing /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/gentoo2020/generic/numpy-1.25.2+computecanada-cp310-cp310-linux_x86_64.whl (from -r requirements_dramv.txt (line 30))
Processing /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/generic/openpyxl-3.1.2+computecanada-py2.py3-none-any.whl (from -r requirements_dramv.txt (line 31))
Processing /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/generic/packaging-23.2+computecanada-py3-none-any.whl (from -r requirements_dramv.txt (line 32))
Processing /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/gentoo/avx2/pandas-2.1.0+computecanada-cp310-cp310-linux_x86_64.whl (from -r requirements_dramv.txt (line 33))
Processing /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/generic/parso-0.8.3+computecanada-py2.py3-none-any.whl (from -r requirements_dramv.txt (line 34))
Processing /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/generic/pexpect-4.8.0+computecanada-py2.py3-none-any.whl (from -r requirements_dramv.txt (line 35))
Processing /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/gentoo2020/avx2/Pillow-10.1.0+computecanada-cp310-cp310-linux_x86_64.whl (from -r requirements_dramv.txt (line 36))
Processing /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/generic/prompt_toolkit-3.0.41+computecanada-py3-none-any.whl (from -r requirements_dramv.txt (line 37))
Processing /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/generic/ptyprocess-0.7.0+computecanada-py2.py3-none-any.whl (from -r requirements_dramv.txt (line 38))
Processing /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/generic/pure_eval-0.2.2+computecanada-py3-none-any.whl (from -r requirements_dramv.txt (line 39))
Processing /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/generic/pygments-2.17.2+computecanada-py3-none-any.whl (from -r requirements_dramv.txt (line 40))
Processing /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/generic/pyparsing-3.0.9+computecanada-py3-none-any.whl (from -r requirements_dramv.txt (line 41))
Processing /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/generic/python_dateutil-2.8.2+computecanada-py2.py3-none-any.whl (from -r requirements_dramv.txt (line 42))
Processing /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/generic/pytz-2023.3.post1+computecanada-py2.py3-none-any.whl (from -r requirements_dramv.txt (line 43))
Processing /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/generic/referencing-0.31.1+computecanada-py3-none-any.whl (from -r requirements_dramv.txt (line 44))
Processing /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/generic/requests-2.31.0+computecanada-py3-none-any.whl (from -r requirements_dramv.txt (line 45))
Processing /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/generic/rpds_py-0.10.0+computecanada-cp310-cp310-linux_x86_64.whl (from -r requirements_dramv.txt (line 46))
Processing /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/generic/scikit_bio-0.5.9+computecanada-cp310-cp310-linux_x86_64.whl (from -r requirements_dramv.txt (line 47))
Processing /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/gentoo/generic/scipy-1.10.1+computecanada-cp310-cp310-linux_x86_64.whl (from -r requirements_dramv.txt (line 48))
Processing /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/generic/six-1.16.0+computecanada-py2.py3-none-any.whl (from -r requirements_dramv.txt (line 49))
Processing /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/generic/SQLAlchemy-2.0.23+computecanada-cp310-cp310-linux_x86_64.whl (from -r requirements_dramv.txt (line 50))
Processing /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/generic/stack_data-0.6.3+computecanada-py3-none-any.whl (from -r requirements_dramv.txt (line 51))
Processing /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/generic/toolz-0.12.0+computecanada-py3-none-any.whl (from -r requirements_dramv.txt (line 52))
Processing /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/generic/traitlets-5.13.0+computecanada-py3-none-any.whl (from -r requirements_dramv.txt (line 53))
Processing /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/generic/typing_extensions-4.8.0+computecanada-py3-none-any.whl (from -r requirements_dramv.txt (line 54))
Processing /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/generic/tzdata-2023.3+computecanada-py2.py3-none-any.whl (from -r requirements_dramv.txt (line 55))
Processing /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/generic/urllib3-2.1.0+computecanada-py3-none-any.whl (from -r requirements_dramv.txt (line 56))
Processing /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/generic/wcwidth-0.2.12+computecanada-py2.py3-none-any.whl (from -r requirements_dramv.txt (line 57))
Installing collected packages: wcwidth, pytz, pure_eval, ptyprocess, urllib3, tzdata, typing_extensions, traitlets, toolz, six, rpds_py, pyparsing, pygments, prompt_toolkit, Pillow, pexpect, parso, packaging, numpy, networkx, natsort, MarkupSafe, kiwisolver, idna, greenlet, fonttools, executing, exceptiongroup, et_xmlfile, decorator, Cython, cycler, charset_normalizer, certifi, attrs, SQLAlchemy, scipy, requests, referencing, python-dateutil, openpyxl, matplotlib_inline, Jinja2, jedi, hdmedians, h5py, contourpy, asttokens, stack_data, pandas, matplotlib, jsonschema_specifications, jsonschema, ipython, scikit_bio, altair, DRAM_bio
Successfully installed Cython-3.0.4+computecanada DRAM_bio-1.4.6+computecanada Jinja2-3.1.2+computecanada MarkupSafe-2.1.3+computecanada Pillow-10.1.0+computecanada SQLAlchemy-2.0.23+computecanada altair-5.1.2+computecanada asttokens-2.4.1+computecanada attrs-23.1.0+computecanada certifi-2023.7.22+computecanada charset_normalizer-3.2.0+computecanada contourpy-1.1.1+computecanada cycler-0.12.1+computecanada decorator-5.1.1+computecanada et_xmlfile-1.1.0+computecanada exceptiongroup-1.1.3+computecanada executing-2.0.1+computecanada fonttools-4.46.0+computecanada greenlet-2.0.2+computecanada h5py-3.8.0+computecanada hdmedians-0.14.2+computecanada idna-3.4+computecanada ipython-8.17.2+computecanada jedi-0.19.1+computecanada jsonschema-4.20.0+computecanada jsonschema_specifications-2023.11.2+computecanada kiwisolver-1.4.5+computecanada matplotlib-3.7.2+computecanada matplotlib_inline-0.1.6+computecanada natsort-8.4.0+computecanada networkx-3.2.1+computecanada numpy-1.25.2+computecanada openpyxl-3.1.2+computecanada packaging-23.2+computecanada pandas-2.1.0+computecanada parso-0.8.3+computecanada pexpect-4.8.0+computecanada prompt_toolkit-3.0.41+computecanada ptyprocess-0.7.0+computecanada pure_eval-0.2.2+computecanada pygments-2.17.2+computecanada pyparsing-3.0.9+computecanada python-dateutil-2.8.2+computecanada pytz-2023.3.post1+computecanada referencing-0.31.1+computecanada requests-2.31.0+computecanada rpds_py-0.10.0+computecanada scikit_bio-0.5.9+computecanada scipy-1.10.1+computecanada six-1.16.0+computecanada stack_data-0.6.3+computecanada toolz-0.12.0+computecanada traitlets-5.13.0+computecanada typing_extensions-4.8.0+computecanada tzdata-2023.3+computecanada urllib3-2.1.0+computecanada wcwidth-0.2.12+computecanada
/localscratch/lauhotte.20827345.0/env/lib/python3.10/site-packages/mag_annotator/database_handler.py:166: UserWarning: Database does not exist at path None
warnings.warn(
/localscratch/lauhotte.20827345.0/env/lib/python3.10/site-packages/mag_annotator/database_handler.py:123: UserWarning: Database does not exist at path None
warnings.warn("Database does not exist at path %s" % description_loc)
2023-12-21 12:18:55,300 - 1 fastas found
2023-12-21 12:18:55,301 - Checking for duplicate names
2023-12-21 12:18:55,340 - No duplicate names found
2023-12-21 12:18:55,340 - Starting the Viral Annotation of Genes with database configuration:
there are no settings, the config is corrupted or too old.
2023-12-21 12:18:55,341 - Retrieved database locations and descriptions
2023-12-21 12:18:55,341 - Annotating final-viral-combined-for-dramv
2023-12-21 12:18:55,565 - Turning genes from prodigal to mmseqs2 db
2023-12-21 12:19:25,273 - No KEGG source provided so distillation will be of limited use.
2023-12-21 12:19:25,276 - Merging ORF annotations
2023-12-21 12:19:25,659 - No rRNAs were detected, no rrnas.tsv file will be created.
2023-12-21 12:19:25,892 - Annotations complete, processing annotations
Traceback (most recent call last):
File "/localscratch/lauhotte.20827345.0/env/bin/DRAM-v.py", line 161, in
We need help to troubleshoot these errors. Thanks for your help.