sos icon indicating copy to clipboard operation
sos copied to clipboard

Possibly better implementation of benchmark workflow

Open gaow opened this issue 6 years ago • 0 comments

Here is how currently benchmark workflows are automatically generated:

https://github.com/gaow/random-nbs/blob/master/slides/20191201_DSC_SoS/sos/test_run.sos

The two key elements in the design are:

  1. Pre-specified input and output file names and group_by
  2. Dependency by sos_step

This will allow running thousands of substeps per step, while still keeping the DAG simple by doing it at step level. But the use of nested workflow is not elegant and seemingly complicated.

gaow avatar Dec 10 '19 15:12 gaow