funannotate
funannotate copied to clipboard
nextgenusfs
funannotate test failing at predict step
Are you using the latest release? funannotate v1.8.17
Describe the bug
I'm trying to run the test after installation using funannotate test -t all --cpus 10 but it crashes in the funannotate predict step
What command did you issue? unannotate test -t all --cpus 10
Logfiles Copying below the output with just the first error which then seems to be repeated 45x, but adding full log file in case necessary.
funannotate test -t all --cpus 10
#########################################################
Running `funannotate clean` unit testing: minimap2 mediated assembly duplications
Downloading: https://osf.io/8pjbe/download?version=1 Bytes: 252076
CMD: funannotate clean -i test.clean.fa -o test.exhaustive.fa --exhaustive
#########################################################
minimap2 version=2.28-r1209 path=/home/pepijn/miniconda3/envs/funannotate/bin/minimap2
-----------------------------------------------
6 input contigs, 6 larger than 500 bp, N50 is 427,039 bp
Checking duplication of 6 contigs
-----------------------------------------------
scaffold_73 appears duplicated: 100% identity over 100% of the contig. contig length: 15153
scaffold_91 appears duplicated: 100% identity over 100% of the contig. contig length: 8858
scaffold_27 appears duplicated: 100% identity over 100% of the contig. contig length: 427039
-----------------------------------------------
6 input contigs; 6 larger than 500 bp; 3 duplicated; 3 written to file
#########################################################
SUCCESS: `funannotate clean` test complete.
#########################################################
#########################################################
Running `funannotate mask` unit testing: RepeatModeler --> RepeatMasker
Downloading: https://osf.io/hbryz/download?version=1 Bytes: 375687
CMD: funannotate mask -i test.fa -o test.masked.fa --cpus 10
#########################################################
-------------------------------------------------------
[Jun 10 02:21 PM]: OS: Ubuntu 22.04, 20 cores, ~ 132 GB RAM. Python: 3.9.19
[Jun 10 02:21 PM]: Running funanotate v1.8.17
[Jun 10 02:21 PM]: Soft-masking simple repeats with tantan
[Jun 10 02:21 PM]: Repeat soft-masking finished:
Masked genome: /home/pepijn/miniconda3/envs/funannotate/bin/test-mask_6d6ef314-2287-415b-905a-b2ccdf6755b2/test.masked.fa
num scaffolds: 2
assembly size: 1,216,048 bp
masked repeats: 50,965 bp (4.19%)
-------------------------------------------------------
#########################################################
SUCCESS: `funannotate mask` test complete.
#########################################################
#########################################################
Running `funannotate predict` unit testing
Downloading: https://osf.io/te2pf/download?version=1 Bytes: 1489808
CMD: funannotate predict -i test.softmasked.fa --protein_evidence protein.evidence.fasta -o annotate --augustus_species saccharomyces --cpus 10 --species Awesome testicus
#########################################################
-------------------------------------------------------
[Jun 10 02:21 PM]: OS: Ubuntu 22.04, 20 cores, ~ 132 GB RAM. Python: 3.9.19
[Jun 10 02:21 PM]: Running funannotate v1.8.17
[Jun 10 02:21 PM]: Skipping CodingQuarry as no --rna_bam passed
[Jun 10 02:21 PM]: Parsed training data, run ab-initio gene predictors as follows:
Program Training-Method
augustus pretrained
genemark selftraining
glimmerhmm busco
snap busco
[Jun 10 02:21 PM]: Loading genome assembly and parsing soft-masked repetitive sequences
[Jun 10 02:21 PM]: Genome loaded: 6 scaffolds; 3,776,588 bp; 19.75% repeats masked
/home/pepijn/miniconda3/envs/funannotate/lib/python3.9/site-packages/funannotate/aux_scripts/funannotate-p2g.py:14: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
from pkg_resources import parse_version
[Jun 10 02:21 PM]: Mapping 1,065 proteins to genome using diamond and exonerate
[Jun 10 02:21 PM]: Found 1,505 preliminary alignments with diamond in 0:00:01 --> generated FASTA files for exonerate in 0:00:00
Progress: 1505 complete, 0 failed, 0 remaining
[Jun 10 02:21 PM]: Exonerate finished in 0:00:21: found 1,272 alignments
[Jun 10 02:21 PM]: Running GeneMark-ES on assembly
[Jun 10 02:24 PM]: 1,569 predictions from GeneMark
[Jun 10 02:24 PM]: Running BUSCO to find conserved gene models for training ab-initio predictors
[Jun 10 02:30 PM]: 370 valid BUSCO predictions found, validating protein sequences
[Jun 10 02:30 PM]: 192 BUSCO predictions validated
[Jun 10 02:30 PM]: Running Augustus gene prediction using saccharomyces parameters
Progress: 11 complete, 0 failed, 0 remaining
[Jun 10 02:32 PM]: 1,485 predictions from Augustus
[Jun 10 02:32 PM]: Pulling out high quality Augustus predictions
[Jun 10 02:32 PM]: Found 371 high quality predictions from Augustus (>90% exon evidence)
[Jun 10 02:32 PM]: Running SNAP gene prediction, using training data: annotate/predict_misc/busco.final.gff3
[Jun 10 02:33 PM]: 0 predictions from SNAP
[Jun 10 02:33 PM]: SNAP prediction failed, moving on without result
[Jun 10 02:33 PM]: Running GlimmerHMM gene prediction, using training data: annotate/predict_misc/busco.final.gff3
[Jun 10 02:33 PM]: 169 predictions from GlimmerHMM
[Jun 10 02:33 PM]: Summary of gene models passed to EVM (weights):
Source Weight Count
Augustus 1 1325
Augustus HiQ 2 372
GeneMark 1 1569
GlimmerHMM 1 169
Total - 3435
[Jun 10 02:33 PM]: EVM: partitioning input to ~ 35 genes per partition using min 1500 bp interval
error: [Errno 2] No such file or directory: '/home/pepijn/miniconda3/envs/funannotate/opt/evidencemodeler-2.1.0/evidence_modeler.pl' run(*(['/home/pepijn/miniconda3/envs/funannotate/opt/evidencemodeler-2.1.0/evidence_modeler.pl', '-G', '/home/pepijn/miniconda3/envs/funannotate/bin/test-predict_6d6ef314-2287-415b-905a-b2ccdf6755b2/annotate/predict_misc/EVM/CP022972.1/CP022972.1_214443-281740/genome.softmasked.fa', '-g', '/home/pepijn/miniconda3/envs/funannotate/bin/test-predict_6d6ef314-2287-415b-905a-b2ccdf6755b2/annotate/predict_misc/EVM/CP022972.1/CP022972.1_214443-281740/gene_predictions.gff3', '-w', '/home/pepijn/miniconda3/envs/funannotate/bin/test-predict_6d6ef314-2287-415b-905a-b2ccdf6755b2/annotate/predict_misc/weights.evm.txt', '--min_intron_length', '10', '--exec_dir', '/home/pepijn/miniconda3/envs/funannotate/bin/test-predict_6d6ef314-2287-415b-905a-b2ccdf6755b2/annotate/predict_misc/EVM/CP022972.1/CP022972.1_214443-281740', '-p', '/home/pepijn/miniconda3/envs/funannotate/bin/test-predict_6d6ef314-2287-415b-905a-b2ccdf6755b2/annotate/predict_misc/EVM/CP022972.1/CP022972.1_214443-281740/protein_alignments.gff3', '/home/pepijn/miniconda3/envs/funannotate/bin/test-predict_6d6ef314-2287-415b-905a-b2ccdf6755b2/annotate/predict_misc/EVM/CP022972.1/CP022972.1_214443-281740/evm.out', '/home/pepijn/miniconda3/envs/funannotate/bin/test-predict_6d6ef314-2287-415b-905a-b2ccdf6755b2/annotate/predict_misc/EVM/CP022972.1/CP022972.1_214443-281740/evm.out.log'],), **{})
Progress: 45 complete, 0 failed, 0 remaining
[Jun 10 02:33 PM]: Converting to GFF3 and collecting all EVM results
[Jun 10 02:33 PM]: Evidence modeler has failed, exiting
#########################################################
Traceback (most recent call last):
File "/home/pepijn/miniconda3/envs/funannotate/bin/funannotate", line 10, in <module>
sys.exit(main())
File "/home/pepijn/miniconda3/envs/funannotate/lib/python3.9/site-packages/funannotate/funannotate.py", line 717, in main
mod.main(arguments)
File "/home/pepijn/miniconda3/envs/funannotate/lib/python3.9/site-packages/funannotate/test.py", line 405, in main
runPredictTest(args)
File "/home/pepijn/miniconda3/envs/funannotate/lib/python3.9/site-packages/funannotate/test.py", line 160, in runPredictTest
assert 1500 <= countGFFgenes(os.path.join(
File "/home/pepijn/miniconda3/envs/funannotate/lib/python3.9/site-packages/funannotate/test.py", line 45, in countGFFgenes
with open(input, 'r') as f:
FileNotFoundError: [Errno 2] No such file or directory: 'test-predict_6d6ef314-2287-415b-905a-b2ccdf6755b2/annotate/predict_results/Awesome_testicus.gff3'
OS/Install Information
- output of
funannotate check --show-versions
-------------------------------------------------------
Checking dependencies for 1.8.17
-------------------------------------------------------
You are running Python v 3.9.19. Now checking python packages...
biopython: 1.79
goatools: 1.4.4
matplotlib: 3.8.4
natsort: 8.4.0
numpy: 1.26.4
pandas: 2.2.2
psutil: 5.9.8
requests: 2.32.3
scikit-learn: 1.2.2
scipy: 1.10.1
seaborn: 0.12.2
All 11 python packages installed
You are running Perl v b'5.032001'. Now checking perl modules...
Carp: 1.50
Clone: 0.46
DBD::SQLite: 1.72
DBD::mysql: 4.050
DBI: 1.643
DB_File: 1.858
Data::Dumper: 2.183
File::Basename: 2.85
File::Which: 1.24
Getopt::Long: 2.54
Hash::Merge: 0.302
JSON: 4.10
LWP::UserAgent: 6.67
Logger::Simple: 2.0
POSIX: 1.94
Parallel::ForkManager: 2.02
Pod::Usage: 1.69
Scalar::Util::Numeric: 0.40
Storable: 3.15
Text::Soundex: 3.05
Thread::Queue: 3.14
Tie::File: 1.06
URI::Escape: 5.17
YAML: 1.30
local::lib: 2.000029
threads: 2.25
threads::shared: 1.61
All 27 Perl modules installed
Checking Environmental Variables...
$FUNANNOTATE_DB=/home/pepijn/funannotate_db
$PASAHOME=/home/pepijn/miniconda3/envs/funannotate/opt/pasa-2.5.3
$TRINITY_HOME=/home/pepijn/miniconda3/envs/funannotate/opt/trinity-2.15.1
$EVM_HOME=/home/pepijn/miniconda3/envs/funannotate/opt/evidencemodeler-2.1.0
$AUGUSTUS_CONFIG_PATH=/home/pepijn/miniconda3/envs/funannotate/config/
$GENEMARK_PATH=/usr/local/bioinf/gmes_linux_64_4
All 6 environmental variables are set
-------------------------------------------------------
Checking external dependencies...
PASA: 2.5.3
CodingQuarry: 2.0
Trinity: 2.15.1
augustus: 3.5.0
bamtools: bamtools 2.5.2
bedtools: bedtools v2.31.1
blat: BLAT v37x1
diamond: 2.1.9
emapper.py: 2.1.12
ete3: 3.1.3
exonerate: exonerate 2.4.0
fasta: 36.3.8g
glimmerhmm: 3.0.4
gmap: 2021-12-17
gmes_petap.pl: 4.71_lic
hisat2: 2.2.1
hmmscan: HMMER 3.4 (Aug 2023)
hmmsearch: HMMER 3.4 (Aug 2023)
java: 22.0.1-internal
kallisto: 0.46.1
mafft: v7.526 (2024/Apr/26)
makeblastdb: makeblastdb 2.15.0+
minimap2: 2.28-r1209
pigz: 2.4
proteinortho: 6.3.1
pslCDnaFilter: no way to determine
salmon: salmon 1.10.3
samtools: samtools 1.18
signalp: 6.0
snap: 2006-07-28
stringtie: 2.2.1
tRNAscan-SE: 2.0.12 (Nov 2022)
tantan: tantan 49
tbl2asn: 25.8
tblastn: tblastn 2.15.0+
trimal: trimAl v1.4.rev15 build[2013-12-17]
trimmomatic: 0.39
All 37 external dependencies are installed
I've had a quick look at the error message error: [Errno 2] No such file or directory: '/home/pepijn/miniconda3/envs/funannotate/opt/evidencemodeler-2.1.0/evidence_modeler.pl' and indeed the perl script is not there, however, it can be found in the following folder: /home/pepijn/miniconda3/envs/funannotate/opt/evidencemodeler-2.1.0/EvmUtils
So it seems to me a simple misdirection to the correct folder. Not sure yet how to correct this but will have a further look
Ok, I went ahead and created a softlink and that solved this problem. Here's the command I used:
ln -s /home/jessica/tools/miniconda3/envs/fun1817/opt/evidencemodeler-2.1.0/EvmUtils/evidence_modeler.pl /home/jessica/tools/miniconda3/envs/fun1817/opt/evidencemodeler-2.1.0/evidence_modeler.pl
Hello, I am having the same challenge. Have you found a solution?
I copied all the .pl files in the lower level folder to make it work. But I feel this needs an adjustment in the code of funannotate. Haven't figured out where though...
Was originally written for EVM v1.1 -- EVMv2 re-organized the code -- I thought I had it working but maybe not. I don't have the time/bandwidth to keep up with all of the dependencies and their constant change. Hence, I started funannotate2 to remove as many dependencies as possible. There shouldn't be a performance different between EVM v2 and v1 -- so can safely just downgrade EVM and should work.
Thanks Jon @nextgenusfs ! For now I fixed by copying the perl scripts, and that seems to work, but good information for anyone else. Any chance you can add a quick not on that in the readthedocs?
And great to hear you're working on a v2, if you need any help testing, I seem to work with weird fungal genomes ;) (are you going to IMC?)
Hi am getting different error please if any one have faced or rectify the error please respond
funannotate predict -i MyAssembly_S8.fa -o predict_Output -s "Colletotrichum sp." --augustus_species fusarium_graminearum --cpus 24 --database /home/aslab3/database/funannotate_db/
[Jul 14 05:52 PM]: OS: Ubuntu 24.04, 32 cores, ~ 330 GB RAM. Python: 3.8.19
[Jul 14 05:52 PM]: Running funannotate v1.8.17
Traceback (most recent call last):
File "/home/aslab3/miniconda3/envs/funannotate/bin/funannotate", line 10, in
