fetchngs
fetchngs copied to clipboard
Running sratools prefetch on Azure
Description of the bug
The exit status of the task that caused the workflow execution to fail was: 1
Error executing process > 'NFCORE_FETCHNGS:SRA:SRA_FASTQ_SRATOOLS:SRATOOLS_PREFETCH (SRR14709033)'
Caused by:
Process `NFCORE_FETCHNGS:SRA:SRA_FASTQ_SRATOOLS:SRATOOLS_PREFETCH (SRR14709033)` terminated with an error exit status (1)
Command executed:
eval "$(vdb-config -o n NCBI_SETTINGS | sed 's/[" ]//g')"
if [[ ! -f "${NCBI_SETTINGS}" ]]; then
mkdir -p "$(dirname "${NCBI_SETTINGS}")"
printf '/LIBS/GUID = "5d0db162-e6a9-4167-a826-10b11c40e7e1"\n/libs/cloud/report_instance_identity = "true"\n' > "${NCBI_SETTINGS}"
fi
retry_with_backoff.sh prefetch \
\
--progress \
SRR14709033
vdb-validate SRR14709033
cat <<-END_VERSIONS > versions.yml
"NFCORE_FETCHNGS:SRA:SRA_FASTQ_SRATOOLS:SRATOOLS_PREFETCH":
sratools: $(prefetch --version 2>&1 | grep -Eo '[0-9.]+')
END_VERSIONS
Command exit status:
1
Command output:
(empty)
Command error:
mkdir: can't create directory '/home/_azbatchtask_1/': Permission denied
Work dir:
az://pipelines/work/b2/8953f0f0c34a6644be0f7ee8a84b21
Tip: you can try to figure out what's wrong by changing to the process work dir and showing the script file named `.command.sh`
Command used and terminal output
Pipeline configuration:
{
"description" : "Pipeline to fetch metadata and raw FastQ files from public databases.",
"launch" : {
"pipeline" : "https://github.com/nf-core/fetchngs",
"revision" : "1.5",
"configProfiles" : [ "test" ],
"paramsText" : "outdir: 'az://pipelines/profile_test/fetchngs/results'"
}
}
``
Relevant files
Configuration used to create Azure environment:
{
"workDir" : "az://pipelines/work",
"region" : "eastus2",
"preRunScript" : "export NXF_VER=21.10.4",
"forge" : {
"vmType" : "Standard_E8ds_v4",
"vmCount" : 60,
"autoScale" : true,
"disposeOnDeletion" : true
},
"deleteJobsOnCompletion" : "on_success",
"deletePoolsOnCompletion" : false,
"environment" : [ ],
"discriminator" : "azure-batch"
}
System information
Nextflow version 21.10.4 Hardware: Azure Container engine: Docker Version of nf-core/fetchngs: 1.5
Maybe users on Azure can define NCBI_SETTINGS
to a writable path? Something like
env.NCBI_SETTINGS = '/common/writable/path/vdb.conf'
I don't have access to facilities to test this.
Given this now works on dev I'd say this issue will be fixed with the next release (v1.10)