issue about panHiTE (Command 'ps' required by nextflow to collect task metrics cannot be found)

[KenanTan-PBP]

trafficstars

Hi,

Here I meet some fatal error when I run the panHiTE.nf by using nextflow. there is the output:

  Loading requirement: jdk/21.0.6

 N E X T F L O W   ~  version 24.10.5

Launching /filer-5/agruppen/PBP/tan/software/HiTE/HiTE/panHiTE.nf [agitated_joliot] DSL2 - revision: 8e7db8af89

executor >  local (1)
[97/93962b] process > pan_preprocess_genomes       [100%] 1 of 1, failed: 1 ✘
[-        ] process > pan_run_hite_single          -
[-        ] process > pan_remove_redundancy        -
[-        ] process > pan_recover_low_copy_TEs     -
[-        ] process > pan_merge_TE_recover         -
[-        ] process > pan_split_genome             -
[-        ] process > pan_annotate_chunk           -
[-        ] process > pan_merge_annotations        -
[-        ] process > pan_recover_split_annotation -
[-        ] process > pan_summarize_tes            -
[-        ] process > pan_gene_te_relation         -
[-        ] process > pan_generate_bam_for_RNA_seq -
[-        ] process > pan_detect_de_genes          -
Execution cancelled -- Finishing pending tasks before exit
ERROR ~ Error executing process > 'pan_preprocess_genomes'

Caused by:
  Process `pan_preprocess_genomes` terminated with an error exit status (1)


Command executed:

  echo "Running preprocessing"
  pan_preprocess_genomes.py --genome_list genome_list --genes_dir gene     --RNA_dir RNA --pan_genomes_dir pan_genomes_dir > preprocess_genomes.log 2>&1

Command exit status:
  1

Command output:
  (empty)

Command error:
  .command.run: line 147: /dev/null: Permission denied
  Command 'ps' required by nextflow to collect task metrics cannot be found
  cp: '.command.out' and '.command.out' are the same file
  cp: '.command.err' and '.command.err' are the same file
  cp: '.command.trace' and '.command.trace' are the same file

Work dir:
  /filer-5/agruppen/PBP/tan/software/HiTE/work/97/93962bab7f1c676d2c1b7c303fffac

Container:
  /filer-5/agruppen/PBP/tan/software/HiTE/HiTE.sif

Tip: you can try to figure out what's wrong by changing to the process work dir and showing the script file named `.command.sh`

 -- Check '.nextflow.log' file for details

there is the script I used:

module load nextflow

. /etc/profile.d/modules.sh

export PATH=$PATH:/usr/bin

source_dir=/filer-5/agruppen/PBP/tan/software/HiTE/HiTE
HiTE_sif_path=/filer-5/agruppen/PBP/tan/software/HiTE/HiTE.sif
pan_genomes_dir=/filer-5/agruppen/PBP/tan/software/HiTE/pan_genomes_dir
singularity_cache=/filer-5/agruppen/PBP/tan/software/HiTE/singularity_cache
genome_list=/filer-5/agruppen/PBP/tan/software/HiTE/genome_list
out_dir=/filer-5/agruppen/PBP/tan/software/HiTE/out_dir
threads=32        

nextflow run ${source_dir}/panHiTE.nf \
 -profile singularity \
 --singularity_name ${HiTE_sif_path} \
 --singularity_cache ${singularity_cache} \
 --pan_genomes_dir ${pan_genomes_dir} \
 --genome_list ${genome_list} \
 --out_dir ${out_dir} \
 --threads ${threads}

I tied to add the -B my/env in nextflow.config file, but nothing changed.

this is the error line in .comand.run. command -v ps &>/dev/null || { >&2 echo "Command 'ps' required by nextflow to collect task metrics cannot be found"; exit 1; } for me, it looks like ps is not properly installed in .sif file? but when I use singularity exec HiTE.sif which ps it shows /bin/ps, which looks normal to me. HiTE.sif is working with command singularity shell or singularity exec. could anyone help me to fix this problem? thank you in advance. if you need more information just leave me a message.

[CSU-KangHu]

I believe this is the main issue:

.command.run: line 147: /dev/null: Permission denied

In panHiTE, many command outputs are redirected to /dev/null.

You can check /filer-5/agruppen/PBP/tan/software/HiTE/work/97/93962bab7f1c676d2c1b7c303fffac/preprocess_genomes.log for more detailed information.

[TKNsama]

I believe this is the main issue:

.command.run: line 147: /dev/null: Permission denied

In panHiTE, many command outputs are redirected to /dev/null.

You can check /filer-5/agruppen/PBP/tan/software/HiTE/work/97/93962bab7f1c676d2c1b7c303fffac/preprocess_genomes.log for more detailed information.

Thanks for your reply! 😊 But why the permission denied. I checked the /dev/null :crw-rw-rw- 1 root root 1, 3 Apr 24 08:42 /dev/null. it looks normal. There is the /dev/null in singularity: Singularity> ls -l /dev/null crw-rw-rw- 1 root root 1, 3 Apr 24 08:42 /dev/null. Actually, I think it is the nextflow problem, maybe I need to give a comment in their git-hub. But still, in case you have some idea, please let me know. Thanks!

By the way, does HiTE support changing the temporary directory? When I try to run Pangenome in parallel, /tmp always fills up.

[CSU-KangHu]

You can install it via Conda, as the Singularity and Docker images may not be updated in a timely manner.
If you install through Conda, you can use the --work_dir option to specify a temporary working directory.

[KenanTan-PBP]

You can install it via Conda, as the Singularity and Docker images may not be updated in a timely manner. If you install through Conda, you can use the --work_dir option to specify a temporary working directory.

Thanks. But in my case, I couldn't install HiTE via conda because conda was banned by my administrator. the only way is to install it by singularity. I will be appreciated if you can updated singularity version when you are free. 😊