dxCompiler icon indicating copy to clipboard operation
dxCompiler copied to clipboard

dxCompiler possibly incorrect localizes input file expressions

Open LiterallyUniqueLogin opened this issue 1 year ago • 11 comments

I'm compiling the following WDL

task load_shared_covars {
  input {
    String script_dir
    File script = "~{script_dir}/traits/load_shared_covars.py"
    File python_array_utils = "~{script_dir}/traits/python_array_utils.py"
  }

  output {
    ...
  }

  command <<< 
    ~{script}
  >>> 

  runtime {
    ...
  }
}

Inside the load_shared_covars.py script I have the line import python_array_utils. However, this fails with the error ModuleNotFoundError: No module named 'python_array_utils'. I'm guessing this is because the python_array_utils input is being mislocalized.

WDL guarantees here that files that originate in the same input directory should be localized into the same runtime directory. So I should be able to rely on being able to import a script that resides in the same input directory. But I'm guessing that dxCompiler isn't properly respecting that for inputs that are assigned to expressions, as here its docs suggest that it does not consider those inputs.

Even if dxCompiler will not considered inputs with default expression values as inputs (which is fine for my use case) can dxCompiler still guarantee that these files get localized to the location that WDL requires?

Happy to provide more details or clarification (or an example run). Also happy to hear if I've misdiagnosed what's going on.

Thanks!

LiterallyUniqueLogin avatar Jan 31 '23 23:01 LiterallyUniqueLogin