cromwell icon indicating copy to clipboard operation
cromwell copied to clipboard

PBSPro Backend Configuration

Open hp2048 opened this issue 5 years ago • 2 comments

Hi We are trying to setup the cromwell + wdl for genomic analyses at the National Computational Infrastructure HPC facility in Australia. This HPC runs bespoke configured PBSPro. I have successfully managed to run "hello world" example workflow using the following configuration for the backend. However, I am unable to modify certain parameters as errors are thrown.

My current configuration is as follows:

   runtime-attributes = """
   Int cpu = 1
   Int memory = 1
   String raijin_queue = "express"
   String walltime = "01:00:00"
   String jobfs = "1GB"
   String raijin_project_id = "myproject"
   """
   #Submit string when there is no "docker" runtime attribute.
   submit = """
   qsub \
   -V \
   -N ${job_name} \
   -o ${out}.qsub \
   -e ${err}.qsub \
   -l ncpus=${cpu} \
   -l mem=${memory}"GB" \
   -l walltime=${walltime} \
   -l jobfs=${jobfs} \
   ${"-q " + raijin_queue} \
   -P ${raijin_project_id} \
   ${script}
   """

My specific questions:

  1. I have tried Float memory_gb = 1.0 as the runtime attribute and ${"-l mem=" + memory_gb + "GB"} as the submit string but this fails with qsub: Illegal attribute or resource value Resource_List.mem error. Could you please help me with the correct formatting of this attribute? I have copied structure of this from SGE.conf.
  2. I would like to use $PROJECT environment variable as the default value for raijin_project_id runtime attribute so that each user can run the same workflow without modification within their allocated project. Is there a way to use environment variable in the config file? I tried ${?PROJECT} and ${PROJECT} as per the recommendations for HOCON but to no avail. I am yet to understand the syntax of HOCON completely to solve this but your help at this time would be much appreciated.
  3. jobfs is a parameter used to control scratch space local to the execution node. Currently it is being passed as a string. Is there a way to convert that into GB same as memory but without the use of keyword memory? Thank you so much for your efforts. Hardip

hp2048 avatar May 13 '19 20:05 hp2048