version 3.0.8 docker: etraining error

[fuesseler]

Good morning, I was trying to switch to the newest version of Braker (v.3.0.8) using the docker container but the pipeline fails at the Etraining step:

ERROR in file /opt/BRAKER/scripts/braker.pl at line 6288 Failed to execute: /opt/Augustus/bin//etraining --species=Sp_56 --AUGUSTUS_CONFIG_PATH=/home/uesseler/miniconda3/pkgs/augustus-3.4.0-pl5321h877ab46_5/config /work/uesseler/docker-braker3-workdir/CtePal_brk3_RNACteno_PROTswissprotodb10vertsaur220324/train.f.gb 1> /work/uesseler/docker-braker3-workdir/CtePal_brk3_RNACteno_PROTswissprotodb10vertsaur220324/gbFilterEtraining.stdout 2>/work/uesseler/docker-braker3-workdir/CtePal_brk3_RNACteno_PROTswissprotodb10vertsaur220324/errors/gbFilterEtraining.stderr I guess the problem might be the double slash here: "/opt/Augustus/bin//etraining" ? I'm not sure if this is a problem with the docker container code, or a problem on my side?

My best regards, F

[KatharinaHoff]

The // is not the problem. What do the stdout and stderr files say?

[fuesseler]

Hi! Could not locate command line parameters file: /home/uesseler/miniconda3/pkgs/augustus-3.4.0-pl5321h877ab46_5/config/parameters/aug_cmdln_parameters.json. I've added this file now to the augustus config directory and running to check if it works now

[tanpham15]

Hi Kathrarina,

Here is my error:

Failed to execute: /opt/Augustus/bin//etraining --species=Sp_4 --AUGUSTUS_CONFIG_PATH=/data/home/mpx586/.augustus /data/scratch/mpx586/Batesia_hypochlora/RNA/braker3/new_braker2/train.f.gb 1> /data/scratch/mpx586/Batesia_hypochlora/RNA/braker3/new_braker2/gbFilterEtraining.stdout 2>/data/scratch/mpx586/Batesia_hypochlora/RNA/braker3/new_braker2/errors/gbFilterEtraining.stderr

In the /braker/errors, they reported some errors:

• In file "samtools.sort.rna_trimmed_minimap2.stderr", [bam_sort_core] merging from 14 files and 7 in-memory blocks... But there is no errors reported in "samtools.sort.rna_trimmed_minimap2.stdout", so as I understand, it was not an error?

• In file "GeneMark-ETP.stderr":

FASTA index file /data/scratch/mpx586/Batesia_hypochlora/RNA/braker3/new_braker2/GeneMark-ETP/data/genome.softmasked.fasta.fai created.
Use of uninitialized value in numeric eq (==) at /opt/ETP/bin/gmes/build_mod.pl line 50.
Use of uninitialized value in numeric eq (==) at /opt/ETP/bin/gmes/build_mod.pl line 56.
04-Apr-24 13:47:58 - INFO: Finding masking penalty maximizing the number of correctly predicted reliable exons in range from 0 to 0.2 with step 0.04
04-Apr-24 13:47:58 - INFO: Running prediction with masking penalty = 0
04-Apr-24 13:53:50 - INFO: Running prediction with masking penalty = 0.04
04-Apr-24 13:59:41 - INFO: Running prediction with masking penalty = 0.08
04-Apr-24 14:05:32 - INFO: Running prediction with masking penalty = 0.12
04-Apr-24 14:11:26 - INFO: Running prediction with masking penalty = 0.16
04-Apr-24 14:17:15 - INFO: Running prediction with masking penalty = 0.2
04-Apr-24 14:23:04 - INFO: Finding masking penalty maximizing the number of correctly predicted reliable exons in range from 0 to 0.04 with step 0.02
04-Apr-24 14:23:04 - INFO: Running prediction with masking penalty = 0.02
04-Apr-24 14:28:57 - INFO: Finding masking penalty maximizing the number of correctly predicted reliable exons in range from 0 to 0.02 with step 0.01
04-Apr-24 14:28:57 - INFO: Running prediction with masking penalty = 0.01
04-Apr-24 14:34:48 - INFO: Selected baseline penalty for the maximum # of correct reliable predictions: 0
04-Apr-24 14:34:48 - INFO: Running prediction with masking penalty = 0.14
04-Apr-24 14:40:38 - INFO: Running prediction with masking penalty = 0.13
04-Apr-24 14:46:31 - INFO: Masking penalty was set to 0.12
Use of uninitialized value in numeric eq (==) at /opt/ETP/bin/gmes/build_mod.pl line 50.
Use of uninitialized value in numeric eq (==) at /opt/ETP/bin/gmes/build_mod.pl line 56.
Use of uninitialized value in numeric eq (==) at /opt/ETP/bin/gmes/build_mod.pl line 50.
Use of uninitialized value in numeric eq (==) at /opt/ETP/bin/gmes/build_mod.pl line 56.
Use of uninitialized value in numeric eq (==) at /opt/ETP/bin/gmes/build_mod.pl line 50.
Use of uninitialized value in numeric eq (==) at /opt/ETP/bin/gmes/build_mod.pl line 56.

• In "GeneMark-ETP.stdout"

....
# warning, skipping short intron
# warning, skipping short intron
# warning, skipping short intron

90554	29485	61069	32.56	prothint/prothint.gff
47878	29485	18393	61.58	r_hints_nonhc.gtf

I am using 8 node * 4 Gb/node, running on command of "braker3 with long read RNA-Seq (only) and protein data"

[KatharinaHoff]

What does it say in /data/scratch/mpx586/Batesia_hypochlora/RNA/braker3/new_braker2/gbFilterEtraining.stdout and /data/scratch/mpx586/Batesia_hypochlora/RNA/braker3/new_braker2/errors/gbFilterEtraining.stderr ?

[tanpham15]

What does it say in /data/scratch/mpx586/Batesia_hypochlora/RNA/braker3/new_braker2/gbFilterEtraining.stdout and /data/scratch/mpx586/Batesia_hypochlora/RNA/braker3/new_braker2/errors/gbFilterEtraining.stderr ?

Hi, When I run both my data and an example of "test3.sh" with braker3_latest.sif, gbFilterEtraining.stderr said: Could not locate command line parameters file: /data/home/mpx586/.augustus/parameters/aug_cmdln_parameters.json. and I didn't get gbFilterEtraining.stderr when running braker3_isoseq.sif

Related to it, there is a warning message related to /data/home/mpx586/.augustus/ on braker.log

# Thu Apr  4 23:25:51 2024: Configuring of BRAKER for using external tools...
# Thu Apr  4 23:25:51 2024: Trying to set $AUGUSTUS_CONFIG_PATH...
# Thu Apr  4 23:25:51 2024: Found environment variable $AUGUSTUS_CONFIG_PATH.
# Thu Apr  4 23:25:51 2024: Checking /opt/Augustus/config/ as potential path for $AUGUSTUS_CONFIG_PATH.
# Thu Apr  4 23:25:51 2024: Success! Setting $AUGUSTUS_CONFIG_PATH to /opt/Augustus/config/!
# Thu Apr  4 23:25:51 2024: WARNING: in file /opt/BRAKER/scripts/braker.pl at line 1931
AUGUSTUS_CONFIG_PATH/species (in this case /opt/Augustus/config//species) is not writeable. BRAKER will try to copy the AUGUSTUS config directory to a writeable location.
# Thu Apr  4 23:25:51 2024: Trying to set $AUGUSTUS_BIN_PATH...
# Thu Apr  4 23:25:51 2024: Found environment variable $AUGUSTUS_BIN_PATH.
# Thu Apr  4 23:25:51 2024: Checking /opt/Augustus/bin/ as potential path for $AUGUSTUS_BIN_PATH.
# Thu Apr  4 23:25:51 2024: Success! Setting $AUGUSTUS_BIN_PATH to /opt/Augustus/bin/!
# Thu Apr  4 23:25:51 2024: Trying to set $AUGUSTUS_SCRIPTS_PATH...
# Thu Apr  4 23:25:51 2024: Found environment variable $AUGUSTUS_SCRIPTS_PATH.
# Thu Apr  4 23:25:51 2024: Checking /opt/Augustus/scripts/ as potential path for $AUGUSTUS_SCRIPTS_PATH.
# Thu Apr  4 23:25:51 2024: Success! Setting $AUGUSTUS_SCRIPTS_PATH to /opt/Augustus/scripts/!
# Thu Apr  4 23:25:51 2024: WARNING: BRAKER will copy the
 AUGUSTUS_CONFIG folder into your home directory!
# Thu Apr  4 23:25:51 2024: WARNING: $AUGUSTUS_CONFIG_PATH/species (in this case /opt/Augustus/config//species ) is not writeable.
*** IMPORTANT: Resetting $AUGUSTUS_CONFIG_PATH=/data/home/mpx586/.augustus because BRAKER requires a writable location!

[spoonbender76]

Hi,

I got the same error in the latest container(singularity build braker3.sif docker://teambraker/braker3:latest) .

gbFilterEtraining.stdout says Could not locate command line parameters file: /home/cnrri01/.augustus/parameters/aug_cmdln_parameters.json.

and gbFilterEtraining.stderr is empty.

UPDATE: I used to run older version of BRAKER singularity container and there was no "parameters" folder in ~/.augustus. The latest container failed to create the "parameters" folder in already existed ~/.augustus path. So I simply removed ~/.augustus folder before a new run and this error did not occur.

[tanpham15]

I am done. 30:45:22 of running, here is the resource I used: 4 nodes * 3Gb/node on HPC, (I got the error when I request 6 nodes).

[KatharinaHoff]

I think these issues are all related to an outdated local augustus config path folder. I have added the solution to this issue to the README.md (start page of the github repository), you only need to delete the local config folder, once. I think it's resolved, right? I will close. Please reopen if I got this wrong.