ERROR while running edta: TE annotation stats results not found

[zx-morningstar]

Hi, after 20 hours, my run crushed when I ran: EDTA.pl --genome ../hbf.fa --sensitive 1 --anno 1 --evaluate 1 --threads 32 --force 1 --overwrite 0 &>edta.log and before crushed, edta reported a lot of lines of warning after " Homology-based annotation of TEs using hbf.fa.mod.EDTA.TElib.fa from scratch.", like: Use of uninitialized value $type in concatenation (.) or string at ~/conda/envs/edta/share/EDTA/util/gff2bed.pl line 84, <GFF> line 5814.

at the end of it, it reported errors: `Evaluate the level of inconsistency for whole-genome TE annotation (slow step):

Use of uninitialized value in pattern match (m//) at ~/conda/envs/edta/share/EDTA/util/call_seq_by_list.pl line 88. Use of uninitialized value $chr_pre in hash element at ~/conda/envs/edta/share/EDTA/util/call_seq_by_list.pl line 90. Use of uninitialized value $pos in pattern match (m//) at ~/conda/envs/edta/share/EDTA/util/call_seq_by_list.pl line 100. Use of uninitialized value $pos in concatenation (.) or string at ~/conda/envs/edta/share/EDTA/util/call_seq_by_list.pl line 103. ERROR: Can not recognize this MSU position in the list! ERROR: TE annotation stats results not found in hbf.fa.mod.EDTA.TE.fa.stat! ` I've been struggling with it for days, could you please tell me how can I solve it? Best wishes!

[oushujun]

Hi,

Have you tried the test file? Please let me know if you encounter any errors with the test run. I need to make sure your installation is complete.

Best, Shujun

[zx-morningstar]

I installed edta with conda, and I just download zip and use the test within it, nohup /usr/bin/time -v perl /db_students1/zx_b17/biosoft/conda/envs/edta/bin/EDTA.pl --genome genome.fa --cds genome.cds.fa --curatedlib ../database/rice6.9.5.liban --exclude genome.exclude.bed --overwrite 1 --sensitive 1 --anno 1 --evaluate 1 --threads 30 > EDTA.test and it seemed successfully ended, because the last line of output show: Exit status: 0 and no errors are reported.

[oushujun]

There are a few things you can try. First, look into the EDTA.test file and see if there are any errors and warnings. If things look good, you may want to simplify your sequence names.

On Sun, Mar 7, 2021 at 3:26 PM zx17 [email protected] wrote:

I installed edta with conda, and I just download zip and use the test within it, nohup /usr/bin/time -v perl /db_students1/zx_b17/biosoft/conda/envs/edta/bin/EDTA.pl --genome genome.fa --cds genome.cds.fa --curatedlib ../database/rice6.9.5.liban --exclude genome.exclude.bed --overwrite 1 --sensitive 1 --anno 1 --evaluate 1 --threads 30 > EDTA.test and it seemed successfully ended, because the last line of output show: Exit status: 0 and no errors are reported.

— You are receiving this because you commented.

Reply to this email directly, view it on GitHub https://github.com/oushujun/EDTA/issues/171#issuecomment-792231767, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABNX4NH7R7U2RV32BFE2X4LTCMTDBANCNFSM4YXM2RJQ .

[zx-morningstar]

There are a few things you can try. First, look into the EDTA.test file and see if there are any errors and warnings. If things look good, you may want to simplify your sequence names. … On Sun, Mar 7, 2021 at 3:26 PM zx17 @.***> wrote: I installed edta with conda, and I just download zip and use the test within it, nohup /usr/bin/time -v perl /db_students1/zx_b17/biosoft/conda/envs/edta/bin/EDTA.pl --genome genome.fa --cds genome.cds.fa --curatedlib ../database/rice6.9.5.liban --exclude genome.exclude.bed --overwrite 1 --sensitive 1 --anno 1 --evaluate 1 --threads 30 > EDTA.test and it seemed successfully ended, because the last line of output show: Exit status: 0 and no errors are reported. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#171 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABNX4NH7R7U2RV32BFE2X4LTCMTDBANCNFSM4YXM2RJQ .

I examine EDTA.test again but everything is fine, and my genome header is like "Chr1" or "Scaffold_34_pilon", I think it's not too complicated, right? BTW, I did use EDTA.pl to deal with this genome several times successfully before, it only went wrong this time with params --sensitive 1 --anno 1 --evaluate 1 --force 1 when it ran "anno", so maybe problems is within "anno" when I didn't use it EDTA will finished successfully.

[a-healey]

Hi Bam and Shujun,

I was about to create a new thread but it seems Bam and I are having the same issue. I received the exact same error using EDTA versions 1.9.4 and 1.9.7. Interestingly, another genome from the same Genus that was assembled and annotated using the same pipeline as the genome that failed finished without any problems. The script used on both genomes was exactly the same --cds cds.fa --sensitive 1 --anno 1 --evaluate 1. Each genome has only a small number of chromosomes and a dozen or so scaffolds (named: scaffold_2, scaffold_24, etc..).

Happy to troubleshoot in any way I can. Thanks for the help,

[oushujun]

Hello @a-healey and @bam2wigKillMe,

It seems like the later updates of the annotation module are not compatible with older annotations. Have you tried removing the entire EDTA.anno folder (or renaming them to keep old results) and rerun EDTA.pl with --anno 1 --step anno? Thanks!

Shujun

[zx-morningstar]

hi,  I just remove the anno dir and rerun the command with params you provided, but it failed exactly like before. best!

---Original--- From: "Shujun Ou"<[email protected]> Date: Tue, Mar 9, 2021 16:49 PM To: "oushujun/EDTA"<[email protected]>; Cc: "zx17"<[email protected]>;"Mention"<[email protected]>; Subject: Re: [oushujun/EDTA] ERROR while running edta: TE annotation stats results not found (#171)

Hello @a-healey and @bam2wigKillMe,

It seems like the later updates of the annotation module are not compatible with older annotations. Have you tried removing the entire EDTA.anno folder (or renaming them to keep old results) and rerun EDTA.pl with --anno 1 --step anno? Thanks!

Shujun

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[a-healey]

Failed for me as well. Empty genome.fa.mod.EDTA.TEanno.sum file, but the program thinks it completed:

Tue Mar 9 08:44:48 CST 2021 TE annotation using the EDTA library has finished! Check out: Whole-genome TE annotation (total TE: ): genome.fa.mod.EDTA.TEanno.gff3 Whole-genome TE annotation summary: genome.fa.mod.EDTA.TEanno.sum Low-threshold TE masking for MAKER gene annotation (masked: 9.04%): genome.fa.mod.MAKER.masked

From the run log: Use of uninitialized value $type in concatenation (.) or string at gff2bed.pl line 84, <GFF> line 4829.

[oushujun]

I rerun the --step anno on the rice genome annotated by v1.9.4 and the pickup run went well with v1.9.7. Did you use an older version to generate the previous steps?

[a-healey]

No, each run was done separately.

[oushujun]

Hi @a-healey @bam2wigKillMe,

I de novo annotated the Arabidopsis genome, with EDTA.pl --genome Arabidopsis_thaliana.TAIR10.dna.toplevel.fa --cds Arabidopsis_thaliana.TAIR10.cds.all.fa --sensitive 1 --anno 1 -t 10 , all is good. You may want to test on this genome too. Also, avoid using --force 1 unless your genome is missing TIR/LTR/Helitron and this parameter is to trick the program to proceed without complete results.

Also, sequence names like "Scaffold_34_pilon" are >15 characters and too long for blastn and you may want to shorten it to something like "Scaffold_34".

Best, Shujun

[conchoecia]

I ran this command and got the same error: TE annotation stats results not found. This was on a fresh install from whatever commit was most current last week.

EDTA.pl --genome /path/to/my/genome.fasta --cds /path/to/my/transcripts.fasta --overwrite 1 --sensitive 1 --anno 1 --evaluate 1 --threads 90

The scaffold names shouldn't be the problem, as they are named, sca1, sca2, sca3..., et cetera. The entire assembly is in 14 scaffolds and the total size is 1.6Gbp.

[oushujun]

There are too many possible reasons. Without a reproducible example, it's very difficult on my end to guessing what may have gone wrong. TE annotation stats results not found. could be the final error message, but previous warnings and errors could provide more info. If none of these can provide a diagnose, you may want to check each directory and see which step failed at first, by comparing files from the test results to your results. - Shujun

[jeremylp2]

I'm also running into this. Has anyone pinpointed a cause?

[oushujun]

@jeremylp2 can you provide the ls -l info from *raw/, *combine/, *final/, and *anno/ folders? Thanks!

[conchoecia]

@oushujun, would it be helpful for you if one of us FTP'd you a directory with our results?

[oushujun]

Yes, having access to data make the diagnose easier. -Shujun

On Tue, Apr 20, 2021 at 1:39 AM darrin t schultz @.***> wrote:

@oushujun https://github.com/oushujun, would it be helpful for you if one of us FTP'd you a directory with our results?

— You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub https://github.com/oushujun/EDTA/issues/171#issuecomment-822650873, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABNX4NHFHNW2O5SVRZSUQQTTJRTFPANCNFSM4YXM2RJQ .

[casett]

I am also having this issue with the latest singularity version. Was a solution found?

[oushujun]

Try the conda version -Shujun

On Sun, May 30, 2021 at 4:37 AM Cassie Ettinger @.***> wrote:

I am also having this issue with the latest singularity version. Was a solution found?

— You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub https://github.com/oushujun/EDTA/issues/171#issuecomment-850895273, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABNX4NGABC52UYZYZ4QUKPTTQFF7BANCNFSM4YXM2RJQ .

[oushujun]

This issue is too old and not sure if it's still valid. Please reopen the issue if the problem is not resolved in the lastest EDTA. Thank you for the support!

Shujun