volcano
volcano copied to clipboard
Is it supported to submit jobs to volcano using WDL language in Cromwell?
The community provides the configuration of using volcano as the cromwell backend, but when the job is submitted, it cannot be successfully delivered to the volcano.
Environment:
- OS:Centos 7.9
- Runtime: Containerd
- Kubernetes version: 1.24.2
- Volcano version: 1.6.0
## https://cromwell.readthedocs.io/en/develop/backends/Volcano/
backend{
default = Volcano
providers{
Volcano {
actor-factory = "cromwell.backend.impl.sfs.config.ConfigBackendLifecycleActorFactory"
config {
runtime-attributes = """
Int runtime_minutes = 600
Int cpus = 2
Int requested_memory_mb_per_core = 8000
String queue = "short"
"""
submit = """
kubectl apply -f ${script}
"""
kill = "vcctl job delete -N ${job_id}"
check-alive = "vcctl job view -N ${job_id}"
job-id-regex = "(\\d+)"
}
}
}
}
test example:
## hw.wdl
task bwa_mem_tool {
String reads_fq1 = "SRR1976948_1.fastq.gz"
String reads_fq2 = "SRR1976948_2.fastq.gz"
String outputdir = "/ags-wdl/bwatest/output"
String fastqFolder = "fastqfolder"
command {
mkdir -p /bwa/${fastqFolder}
cd /bwa/${fastqFolder}
rm -rf SRR1976948*
wget https://ags-public.oss-cn-beijing.aliyuncs.com/alignment/subset_assembly.fa.gz
wget https://ags-public.oss-cn-beijing.aliyuncs.com/alignment/SRR1976948_1.fastq.gz
wget https://ags-public.oss-cn-beijing.aliyuncs.com/alignment/SRR1976948_2.fastq.gz
gzip -cd ${reference} > subset_assembly.fa
gunzip -c ${reads_fq1} | head -800000 > SRR1976948.1
gunzip -c ${reads_fq2} | head -800000 > SRR1976948.2
bwa index subset_assembly.fa
bwa aln subset_assembly.fa SRR1976948.1 > ${outputdir}/SRR1976948.1.untrimmed.sai
bwa aln subset_assembly.fa SRR1976948.2 > ${outputdir}/SRR1976948.2.untrimmed.sai
}
output {
File sam1 = "${outputdir}/SRR1976948.1.untrimmed.sai"
File sam2 = "${outputdir}/SRR1976948.2.untrimmed.sai"
}
runtime {
docker: "biocontainers/bwa:v0.7.17_cv1"
memory: "2GB"
cpu: 1
}
}
workflow wf {
call bwa_mem_tool
}
java -Dconfig.file=./volcano.conf -jar cromwell-83.jar run hw.wdl
What I want to know:
Is it supported to submit jobs to volcano using WDL language in Cromwell?If supported, can you provide relevant cases for users?If not, when will it be supported in the future?