Bo

> there will be no distinction between resources used by different steps, unless as you said there is a way to honor those specifications? Yes, because in that case all...

> what would be the advantage to use task over non-task execution In my case tasks are "independent" of SoS processes so my notebook server would be killed/closed with running...

Another reason is resource management, as this ticket represents. A large workflow can consist of large and small steps and with varying requirements for resources. Tasks allow us to submit...

Even more important might be interactive data analysis since I usually do not have a complete workflow at hand to run, rather a sequence of steps that need to be...

`-r` works in a totally different way. It does not use `task: mem=.... ` and its status cannot be monitored by `sos status`. All it does are 1. Copy the...

What this ticket talks about is that currently sos workers takes substeps regardless of the "size" of substeps. So if we are using 8 workers, 4 each on 2 hosts,...

Without 1, we cannot really prevent the cited problem by comparing "requested resource" (known) and "available resource" (unknown), but we can potentially do better load balancing. That is to say,...

> trunk_size and trunk_workers in the non-task context can be useful in specifying how many substeps goes into one file, You idea is trying to group small substeps so that...

`trunk_size` is not relevant in this case. For every substep, we have, for example, a single R-script. With task, it will be a task. With `trunk_size`, it will be a...

For `-r`, you only get one job submission for a single `sos run` command, which is likely a multi-node multi-core job that executes all substeps.