sos
sos copied to clipboard
Possibly better implementation of benchmark workflow
Here is how currently benchmark workflows are automatically generated:
https://github.com/gaow/random-nbs/blob/master/slides/20191201_DSC_SoS/sos/test_run.sos
The two key elements in the design are:
- Pre-specified input and output file names and
group_by - Dependency by
sos_step
This will allow running thousands of substeps per step, while still keeping the DAG simple by doing it at step level. But the use of nested workflow is not elegant and seemingly complicated.