Support job depencencies

[mllg]

https://hpc.nih.gov/docs/job_dependencies.html

[jgrn307]

I think this would be great. You could create/submit the jobs, and then when all of the jobs are completed have a "merge" job trigger without having the "submitter" job idling.

[wlandau]

Yes, thanks @mllg! This could make drake a lot more powerful, especially if future.batchtools adopts the functionality too. cc @HenrikBengtsson

[jgrn307]

By the way, I'm happy to help test this out on a SLURM machine. It seems a good first step would be to focus on batchtools combine-type functions.

[jgrn307]

Small correction with the above link -- when submitting the job via slurm, you will want to sbatch using the --parsable flag, e.g.: sbatch --parsable [blah blah] This will return ONLY the jobid and (optionally) the cluster name separated by a semicolon (the jobid is what you will need for job dependencies in SLURM -- if the semicolon is not in the string than the entire string should be the jobid). Without it, different slurm systems may output different text. We tried the NIH link above initially and it didn't work due to the lack of the --parsable, we had it return something like "Your job ID is [jobid]", the text of which may differ from system to system.

See: https://slurm.schedmd.com/sbatch.html