volcano icon indicating copy to clipboard operation
volcano copied to clipboard

Is it supported to submit jobs to volcano using WDL language in Cromwell?

Open chloe6888 opened this issue 3 years ago • 0 comments

The community provides the configuration of using volcano as the cromwell backend, but when the job is submitted, it cannot be successfully delivered to the volcano.

Environment:

  • OS:Centos 7.9
  • Runtime: Containerd
  • Kubernetes version: 1.24.2
  • Volcano version: 1.6.0
## https://cromwell.readthedocs.io/en/develop/backends/Volcano/
backend{
  default = Volcano
  providers{
    Volcano {
      actor-factory = "cromwell.backend.impl.sfs.config.ConfigBackendLifecycleActorFactory"
      config {
        runtime-attributes = """
        Int runtime_minutes = 600
        Int cpus = 2
        Int requested_memory_mb_per_core = 8000
        String queue = "short"
        """
        submit = """
            kubectl apply -f ${script}
        """
        kill = "vcctl job delete -N ${job_id}"
        check-alive = "vcctl job view -N ${job_id}"
        job-id-regex = "(\\d+)"
      }
    }
  }
}

test example:

## hw.wdl
task bwa_mem_tool {
  String reads_fq1 = "SRR1976948_1.fastq.gz"
  String reads_fq2 = "SRR1976948_2.fastq.gz"
  String outputdir = "/ags-wdl/bwatest/output"
  String fastqFolder = "fastqfolder"
  command {
        mkdir -p /bwa/${fastqFolder}
        cd /bwa/${fastqFolder}
        rm -rf SRR1976948*
        wget https://ags-public.oss-cn-beijing.aliyuncs.com/alignment/subset_assembly.fa.gz
        wget https://ags-public.oss-cn-beijing.aliyuncs.com/alignment/SRR1976948_1.fastq.gz
        wget https://ags-public.oss-cn-beijing.aliyuncs.com/alignment/SRR1976948_2.fastq.gz
        gzip -cd ${reference} > subset_assembly.fa
        gunzip -c ${reads_fq1} | head -800000 > SRR1976948.1
        gunzip -c ${reads_fq2} | head -800000 > SRR1976948.2
        bwa index subset_assembly.fa
        bwa aln subset_assembly.fa SRR1976948.1 > ${outputdir}/SRR1976948.1.untrimmed.sai
        bwa aln subset_assembly.fa SRR1976948.2 > ${outputdir}/SRR1976948.2.untrimmed.sai
  }
  output {
    File sam1 = "${outputdir}/SRR1976948.1.untrimmed.sai"
    File sam2 = "${outputdir}/SRR1976948.2.untrimmed.sai"
  }
  runtime {
    docker: "biocontainers/bwa:v0.7.17_cv1"
    memory: "2GB"
    cpu: 1
  }
}
workflow wf {
  call bwa_mem_tool
}


java -Dconfig.file=./volcano.conf -jar cromwell-83.jar run hw.wdl

What I want to know:

Is it supported to submit jobs to volcano using WDL language in Cromwell?If supported, can you provide relevant cases for users?If not, when will it be supported in the future?

chloe6888 avatar Sep 21 '22 06:09 chloe6888