AlgTask Error

[gottsc33]

It appears that is an issue with the AlgTask component of Clustalo. The following terminal output is from a run using a previous dataset that was functioning 2 months ago.

INFO -  Testing x86-64  portable applications...
       clustalo: OK - 1.2.4
      dialigntx: OK - This is DIALIGN-TX Version 1.0.2 - A Multiple Sequence alignment program.
       fasttree: OK - Usage for FastTree version 2.1.8 Double precision (No SSE3), OpenMP (1 threads):
         iqtree: OK - IQ-TREE multicore version 1.5.5 for Linux 64-bit built Jun  2 2017
         kalign: OK - Kalign version 2.04, Copyright (C) 2004, 2005, 2006 Timo Lassmann
          mafft: OK - MAFFT v6.861b (2011/09/24)
         muscle: OK - MUSCLE v3.8.31 by Robert C. Edgar
          phyml: OK - . This is PhyML version 20160115.
     pmodeltest: OK - pmodeltest.py v1.4
          raxml: OK - This is RAxML version 8.2.11 released by Alexandros Stamatakis on June 2017.
 raxml-pthreads: OK - This is RAxML version 8.2.11 released by Alexandros Stamatakis on June 2017.
         readal: OK - readAl v1.4.rev6 build[2012-02-02]
         statal: OK - statAl v1.4.rev6 build[2012-02-02]
        tcoffee: MISSING
         trimal: OK - trimAl v1.4.rev6 build[2012-02-02]

WRNG -  68 target sequences
INFO -  ETE build starts now!
INFO -   Updating tasks status: (Thu Mar 25 14:48:01 2021)
INFO -  Thread clustalo_default-none-none-fasttree_full: pending tasks: 1 of sizes: 68
INFO -  
INFO -   (W) MultiSeqTask (68 aa seqs, MSF, /clustalo_d...ttree_full)
INFO -     0 jobs: 
INFO -     TaskDir: bf148c3bdd7b824f32a05f44d2397154
INFO -  Launched 0 jobs. 0(R), 0(W). Cores usage: 0/1
INFO -     Processing done task: MultiSeqTask (68 aa seqs, MSF, /clustalo_d...ttree_full)
INFO -     
INFO -   (D) MultiSeqTask (68 aa seqs, MSF, /clustalo_d...ttree_full)
INFO -     0 jobs: 1(D)
INFO -     TaskDir: bf148c3bdd7b824f32a05f44d2397154
INFO -          Registering new task: AlgTask (68 aa seqs, Clustal-Omega, /?)
INFO -  Waiting 2 seconds
INFO -  
INFO -   Updating tasks status: (Thu Mar 25 14:48:03 2021)
INFO -  Thread clustalo_default-none-none-fasttree_full: pending tasks: 1 of sizes: 68
INFO -  
INFO -   (W) AlgTask (68 aa seqs, Clustal-Omega, /clustalo_d...ttree_full)
INFO -     1 jobs: 
INFO -     TaskDir: 94d1d9bb6d07d7553b04154588e59ead
INFO -       (None): Job (clustalo---threads-1, af9e4a)
INFO -       Queueing  Job (clustalo---threads-1, af9e4a) from AlgTask (68 aa seqs, Clustal-Omega, /clustalo_d...ttree_full)
INFO -  Waiting 2 seconds
INFO -  Launched 1 jobs. 1(R), 0(W). Cores usage: 1/1
INFO -  
INFO -   Updating tasks status: (Thu Mar 25 14:48:05 2021)
INFO -  Thread clustalo_default-none-none-fasttree_full: pending tasks: 1 of sizes: 68
INFO -  
INFO -   (W) AlgTask (68 aa seqs, Clustal-Omega, /clustalo_d...ttree_full)
INFO -     1 jobs: 1(W)
INFO -     TaskDir: 94d1d9bb6d07d7553b04154588e59ead
INFO -       (Q): Job (clustalo---threads-1, af9e4a)
INFO -  Waiting 2 seconds
LOST PROCESS <subprocess.Popen object at 0x2b6aee21e518> af9e4ade0e7f405efd03f72796d97990
INFO -  Launched 0 jobs. 0(R), 0(W). Cores usage: 0/1
INFO -  
INFO -   Updating tasks status: (Thu Mar 25 14:48:07 2021)
INFO -  Thread clustalo_default-none-none-fasttree_full: pending tasks: 1 of sizes: 68
INFO -  
INFO -   (R) AlgTask (68 aa seqs, Clustal-Omega, /clustalo_d...ttree_full)
INFO -     1 jobs: 1(R)
INFO -     TaskDir: 94d1d9bb6d07d7553b04154588e59ead
INFO -       (R): Job (clustalo---threads-1, af9e4a)
ERR  -        Job error reported: Job (clustalo---threads-1, af9e4a)
ERR  -        Errors found in AlgTask (68 aa seqs, Clustal-Omega, /clustalo_d...ttree_full)
Traceback (most recent call last):
  File "anaconda3/lib/python3.6/site-packages/ete3/tools/ete_build_lib/scheduler.py", line 257, in schedule
    task.status = task.get_status(qstat_jobs)
  File "anaconda3/lib/python3.6/site-packages/ete3/tools/ete_build_lib/master_task.py", line 198, in get_status
    self.job_status = self.get_jobs_status(sge_jobs)
  File "anaconda3/lib/python3.6/site-packages/ete3/tools/ete_build_lib/master_task.py", line 306, in get_jobs_status
    raise TaskError(j, "Job execution error %s" %errorpath)
ete3.tools.ete_build_lib.errors.TaskError: Job execution error ~/phylogeny_output/test/4/tasks/af9e4ade0e7f405efd03f72796d97990
INFO -  Waiting 2 seconds
ERR  -  Thread clustalo_default-none-none-fasttree_full contains errors:
ERR  -   ** AlgTask (68 aa seqs, Clustal-Omega, /clustalo_d...ttree_full)
ERR  -        -> Job (clustalo---threads-1, af9e4a)
ERR  -        -> ~/phylogeny_output/test/4/tasks/af9e4ade0e7f405efd03f72796d97990
ERR  -          -> Job execution error ~/phylogeny_output/test/4/tasks/af9e4ade0e7f405efd03f72796d97990
INFO -  
ERR  -  Done with ERRORS

Data Error: Errors found in some tasks

It's running on a CentOS system, installed using conda, with python 3.6.8. I tried reinstalling ETE3 on an ubuntu system with python versions 2.7 and 3.4 and the same error is occurring.

[jhcepas]

Hi, could you check the log files in the task folder? ~ /phylogeny_output/test/4/tasks/af9e4ade0e7f405efd03f72796d97990 It could be a problem with the data format expected by Clustal Omega... (i.e. wrong format, illegal characters, etc)

On Fri, 26 Mar 2021 at 21:49, Christopher Gottschalk < @.***> wrote:

It appears that is an issue with the AlgTask component of Clustalo. The following terminal output is from a run using a previous dataset that was functioning 2 months ago.

INFO - Testing x86-64 portable applications... clustalo: OK - 1.2.4 dialigntx: OK - This is DIALIGN-TX Version 1.0.2 - A Multiple Sequence alignment program. fasttree: OK - Usage for FastTree version 2.1.8 Double precision (No SSE3), OpenMP (1 threads): iqtree: OK - IQ-TREE multicore version 1.5.5 for Linux 64-bit built Jun 2 2017 kalign: OK - Kalign version 2.04, Copyright (C) 2004, 2005, 2006 Timo Lassmann mafft: OK - MAFFT v6.861b (2011/09/24) muscle: OK - MUSCLE v3.8.31 by Robert C. Edgar phyml: OK - . This is PhyML version 20160115. pmodeltest: OK - pmodeltest.py v1.4 raxml: OK - This is RAxML version 8.2.11 released by Alexandros Stamatakis on June 2017. raxml-pthreads: OK - This is RAxML version 8.2.11 released by Alexandros Stamatakis on June 2017. readal: OK - readAl v1.4.rev6 build[2012-02-02] statal: OK - statAl v1.4.rev6 build[2012-02-02] tcoffee: MISSING trimal: OK - trimAl v1.4.rev6 build[2012-02-02]

WRNG - 68 target sequences INFO - ETE build starts now! INFO - Updating tasks status: (Thu Mar 25 14:48:01 2021) INFO - Thread clustalo_default-none-none-fasttree_full: pending tasks: 1 of sizes: 68 INFO - INFO - (W) MultiSeqTask (68 aa seqs, MSF, /clustalo_d...ttree_full) INFO - 0 jobs: INFO - TaskDir: bf148c3bdd7b824f32a05f44d2397154 INFO - Launched 0 jobs. 0(R), 0(W). Cores usage: 0/1 INFO - Processing done task: MultiSeqTask (68 aa seqs, MSF, /clustalo_d...ttree_full) INFO - INFO - (D) MultiSeqTask (68 aa seqs, MSF, /clustalo_d...ttree_full) INFO - 0 jobs: 1(D) INFO - TaskDir: bf148c3bdd7b824f32a05f44d2397154 INFO - Registering new task: AlgTask (68 aa seqs, Clustal-Omega, /?) INFO - Waiting 2 seconds INFO - INFO - Updating tasks status: (Thu Mar 25 14:48:03 2021) INFO - Thread clustalo_default-none-none-fasttree_full: pending tasks: 1 of sizes: 68 INFO - INFO - (W) AlgTask (68 aa seqs, Clustal-Omega, /clustalo_d...ttree_full) INFO - 1 jobs: INFO - TaskDir: 94d1d9bb6d07d7553b04154588e59ead INFO - (None): Job (clustalo---threads-1, af9e4a) INFO - Queueing Job (clustalo---threads-1, af9e4a) from AlgTask (68 aa seqs, Clustal-Omega, /clustalo_d...ttree_full) INFO - Waiting 2 seconds INFO - Launched 1 jobs. 1(R), 0(W). Cores usage: 1/1 INFO - INFO - Updating tasks status: (Thu Mar 25 14:48:05 2021) INFO - Thread clustalo_default-none-none-fasttree_full: pending tasks: 1 of sizes: 68 INFO - INFO - (W) AlgTask (68 aa seqs, Clustal-Omega, /clustalo_d...ttree_full) INFO - 1 jobs: 1(W) INFO - TaskDir: 94d1d9bb6d07d7553b04154588e59ead INFO - (Q): Job (clustalo---threads-1, af9e4a) INFO - Waiting 2 seconds LOST PROCESS <subprocess.Popen object at 0x2b6aee21e518> af9e4ade0e7f405efd03f72796d97990 INFO - Launched 0 jobs. 0(R), 0(W). Cores usage: 0/1 INFO - INFO - Updating tasks status: (Thu Mar 25 14:48:07 2021) INFO - Thread clustalo_default-none-none-fasttree_full: pending tasks: 1 of sizes: 68 INFO - INFO - (R) AlgTask (68 aa seqs, Clustal-Omega, /clustalo_d...ttree_full) INFO - 1 jobs: 1(R) INFO - TaskDir: 94d1d9bb6d07d7553b04154588e59ead INFO - (R): Job (clustalo---threads-1, af9e4a) ERR - Job error reported: Job (clustalo---threads-1, af9e4a) ERR - Errors found in AlgTask (68 aa seqs, Clustal-Omega, /clustalo_d...ttree_full) Traceback (most recent call last): File "anaconda3/lib/python3.6/site-packages/ete3/tools/ete_build_lib/scheduler.py", line 257, in schedule task.status = task.get_status(qstat_jobs) File "anaconda3/lib/python3.6/site-packages/ete3/tools/ete_build_lib/master_task.py", line 198, in get_status self.job_status = self.get_jobs_status(sge_jobs) File "anaconda3/lib/python3.6/site-packages/ete3/tools/ete_build_lib/master_task.py", line 306, in get_jobs_status raise TaskError(j, "Job execution error %s" %errorpath) ete3.tools.ete_build_lib.errors.TaskError: Job execution error ~/phylogeny_output/test/4/tasks/af9e4ade0e7f405efd03f72796d97990 INFO - Waiting 2 seconds ERR - Thread clustalo_default-none-none-fasttree_full contains errors: ERR - ** AlgTask (68 aa seqs, Clustal-Omega, /clustalo_d...ttree_full) ERR - -> Job (clustalo---threads-1, af9e4a) ERR - -> ~/phylogeny_output/test/4/tasks/af9e4ade0e7f405efd03f72796d97990 ERR - -> Job execution error ~/phylogeny_output/test/4/tasks/af9e4ade0e7f405efd03f72796d97990 INFO - ERR - Done with ERRORS

Data Error: Errors found in some tasks

It's running on a CentOS system, installed using conda, with python 3.6.8. I tried reinstalling ETE3 on an ubuntu system with python versions 2.7 and 3.4 and the same error is occurring.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/etetoolkit/ete/issues/497, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABH6STEY5SPDYQDHAUDMZDTFTXN7ANCNFSM4Z4CJ7KA .

[gottsc33]

There were two output files. The cmd output contained the following lines: #!/bin/sh (echo R > ~/phylogeny_output/test/4/tasks/af9e4ade0e7f405efd03f72796d97990/__status__ && date +'%a %b %d %H:%M:%S %Y' > ~/phylogeny_output/test/4/tasks/af9e4ade0e7f405efd03f72796d97990__time__) && (cd ~/phylogeny_output/test/4/tasks/af9e4ade0e7f405efd03f72796d97990 && /home/chris/anaconda3/envs/ete3-env/bin/ete3_apps/bin/clustalo --threads 1 -i ~/phylogeny_output/test/4/input/bf148c3bdd7b824f32a05f44d2397154.100 -o clustalo_alg.fasta --outfmt fa && (echo D > ~/phylogeny_output/test/4/tasks/af9e4ade0e7f405efd03f72796d97990/__status__; ) || (echo E > ~/phylogeny_output/test/4/tasks/af9e4ade0e7f405efd03f72796d97990/__status__; )); date +'%a %b %d %H:%M:%S %Y' >> ~/phylogeny_output/test/4/tasks/af9e4ade0e7f405efd03f72796d97990/__time__;

The other file status only contained the letter "E".