TransPi on HPC failing to pull singularity image

[claumer]

Hi there,

I've been trying to use TransPi (love the concept and the benchmarking in the paper looks great!) to re-assemble some older public datasets. Running on an interactive slurm session of an HPC which has singularity installed:

---

(TransPi) chril1@hpc-gpu-002:~/TransPiDBs$ ./nextflow run ../TransPi/TransPi.nf --all --maxReadLen 150 --k 25,35,55,75,85 --reads '/home/chril1/NERC_PhyloCourse/transcriptome_data/Flagellophora_sp/SRR8641368_R[1,2].fastq.gz' --outdir Results_Flagellophora -profile singularity,TransPiContainer

N E X T F L O W ~ version 21.04.1 Launching ../TransPi/TransPi.nf [furious_einstein] - revision: 5a26a6e85d

TransPi - Transcriptome Analysis Pipeline v1.3.0-rc

TransPi.nf Directory: /gpfs/nhmfsa/bulk/share/data/mbl/share/workspaces/users/chril1/TransPi/TransPi.nf Launch Directory: /gpfs/nhmfsa/bulk/share/data/mbl/share/workspaces/users/chril1/TransPiDBs Results Directory: Results_Flagellophora Work Directory: /gpfs/nhmfsa/bulk/share/data/mbl/share/workspaces/users/chril1/TransPiDBs/work TransPi DBs: /home/chril1/TransPiDBs Uniprot DB: /home/chril1/TransPiDBs/DBs/uniprot_db/uniprot_metazoa_33208.fasta Busco DB: /home/chril1/TransPiDBs/DBs/busco_db/metazoa_odb10 Reads Directory: /home/chril1/NERC_PhyloCourse/transcriptome_data/Flagellophora_sp/SRR8641368_R[1,2].fastq.gz Read Length: 150 Kmers: 25,35,55,75,85

Running TransPi with your dataset


Running the full TransPi analysis

executor > local (2) [6c/876909] process > fasqc (SRR8641368_R) [ 0%] 0 of 1 executor > local (3) [6c/876909] process > fasqc (SRR8641368_R) [ 0%] 0 of 1 executor > local (3) [6c/876909] process > fasqc (SRR8641368_R) [ 0%] 0 of 1 executor > local (3) [6c/876909] process > fasqc (SRR8641368_R) [ 0%] 0 of 1 executor > local (3) [6c/876909] process > fasqc (SRR8641368_R) [ 0%] 0 of 1 executor > local (3) [6c/876909] process > fasqc (SRR8641368_R) [100%] 1 of 1 ✔ executor > local (3) [6c/876909] process > fasqc (SRR8641368_R) [100%] 1 of 1 ✔ executor > local (4) executor > local (5) executor > local (6) executor > local (6) executor > local (6) executor > local (6) executor > local (6) executor > local (6) [6c/876909] process > fasqc (SRR8641368_R) [100%] 1 of 1 ✔ executor > local (8) [6c/876909] process > fasqc (SRR8641368_R) [100%] 1 of 1 ✔ executor > local (9) [6c/876909] process > fasqc (SRR8641368_R) [100%] 1 of 1 ✔ executor > local (9) [6c/876909] process > fasqc (SRR8641368_R) [100%] 1 of 1 ✔ executor > local (10) executor > local (10) executor > local (10) executor > local (10) executor > local (11) executor > local (11) executor > local (11) executor > local (11) executor > local (11) [6c/876909] process > fasqc (SRR8641368_R) [100%] 1 of 1 ✔ executor > local (13) [6c/876909] process > fasqc (SRR8641368_R) [100%] 1 of 1 ✔ executor > local (13) [6c/876909] process > fasqc (SRR8641368_R) [100%] 1 of 1 ✔ [38/2daf75] process > fastp (SRR8641368_R) [100%] 1 of 1 ✔ [09/b36296] process > fastp_stats (SRR8641368_R) [100%] 1 of 1 ✔ [a5/77f26f] process > skip_rrna_removal (SRR8641368_R) [100%] 1 of 1 ✔ [94/71880f] process > normalize_reads (SRR8641368_R) [100%] 1 of 1 ✔ [de/71f78c] process > trinity_assembly (SRR8641368_R) [100%] 1 of 1 ✔ [de/bc6216] process > soap_assembly (SRR8641368_R) [100%] 1 of 1 ✔ [25/457622] process > velvet_oases_assembly (SRR8641368_R) [100%] 1 of 1 ✔ [84/17fe6c] process > rna_spades_assembly (SRR8641368_R) [100%] 1 of 1 ✔ [2e/672733] process > transabyss_assembly (SRR8641368_R) [100%] 1 of 1 ✔ [80/f0f8e6] process > evigene (SRR8641368_R) [ 0%] 0 of 1 [- ] process > rna_quast - [- ] process > mapping_evigene - [- ] process > busco4 - [ca/050e8b] process > mapping_trinity (SRR8641368_R) [ 0%] 0 of 1 [- ] process > summary_evigene_individual - [- ] process > busco4_tri - [47/a485e7] process > skip_busco_dist (SRR8641368_R) [100%] 1 of 1, cached: 1 ✔ executor > local (13) [6c/876909] process > fasqc (SRR8641368_R) [100%] 1 of 1 ✔ [38/2daf75] process > fastp (SRR8641368_R) [100%] 1 of 1 ✔ [09/b36296] process > fastp_stats (SRR8641368_R) [100%] 1 of 1 ✔ [a5/77f26f] process > skip_rrna_removal (SRR8641368_R) [100%] 1 of 1 ✔ [94/71880f] process > normalize_reads (SRR8641368_R) [100%] 1 of 1 ✔ [de/71f78c] process > trinity_assembly (SRR8641368_R) [100%] 1 of 1 ✔ [de/bc6216] process > soap_assembly (SRR8641368_R) [100%] 1 of 1 ✔ [25/457622] process > velvet_oases_assembly (SRR8641368_R) [100%] 1 of 1 ✔ [84/17fe6c] process > rna_spades_assembly (SRR8641368_R) [100%] 1 of 1 ✔ [2e/672733] process > transabyss_assembly (SRR8641368_R) [100%] 1 of 1 ✔ [80/f0f8e6] process > evigene (SRR8641368_R) [ 0%] 0 of 1 [- ] process > rna_quast - [- ] process > mapping_evigene - [- ] process > busco4 - [ca/050e8b] process > mapping_trinity (SRR8641368_R) [ 0%] 0 of 1 [- ] process > summary_evigene_individual - [- ] process > busco4_tri - [47/a485e7] process > skip_busco_dist (SRR8641368_R) [100%] 1 of 1, cached: 1 ✔ [- ] process > summary_busco4_individual - [- ] process > get_busco4_comparison - [- ] process > transdecoder_longorf - [- ] process > transdecoder_diamond - [- ] process > transdecoder_hmmer - [- ] process > transdecoder_predict - [- ] process > swiss_diamond_trinotate - [- ] process > custom_diamond_trinotate - [- ] process > hmmer_trinotate - [- ] process > skip_signalP - [- ] process > skip_tmhmm - [- ] process > skip_rnammer - [- ] process > trinotate - [- ] process > get_GO_comparison - [- ] process > summary_custom_uniprot - [- ] process > skip_kegg - [- ] process > get_transcript_dist - [- ] process > summary_transdecoder_individual - [- ] process > summary_trinotate_individual - [- ] process > get_report - [98/934751] process > get_run_info [100%] 1 of 1 ✔ Pulling Singularity image docker://null [cache /gpfs/nhmfsa/bulk/share/data/mbl/share/workspaces/users/chril1/TransPiDBs/singularityCache/null.img] Error executing process > 'busco4_tri (SRR8641368_R)'

Caused by: Failed to pull singularity image command: singularity pull --name null.img.pulling.1664887974429 docker://null > /dev/null status : 255 message: FATAL: While making image from oci registry: error fetching image to cache: failed to get checksum for docker://null: Error reading manifest latest in docker.io/library/null: errors: denied: requested access to the resource is denied unauthorized: authentication required

---

It seems that it gets all the way through building the four assemblies then crashes when trying to pull a singularity image for BUSCO evaluation. I confess I'm not quite sure how to debug this one - do you have any ideas? I've also tried -profile conda and am getting environment creation conflict errors out of that, and so seem to have reached an impasse.

Thanks very much for your attention,

Chris L