sos
sos copied to clipboard
vatlab
SoS workflow system for daily data analysis
Right now we allow for ``` dynamic('a.txt') dynamic(['a.txt', 'b.txt']) dynamic('*.txt') ``` and we resolve them to ``` 'a.txt' 'a.txt', 'b.txt' pattern matching.... '*.txt' ``` and the returned target might not...
``` $ sos remote setup 172.16.209.130 WARNING: Treating undefined host 172.16.209.130 as address of a remote host. INFO: Using existing public key .ssh/id_rsa.pub ERROR: 'Namespace' object has no attribute 'test_connection'...
I've got in my simulation runs: ``` WARNING: Task M20_b38d872198d6c6d4 inactive for more than 85 seconds, might have been killed. INFO: 19 of 20 tasks in M20_b38d872198d6c6d4 failed ``` But...
#1378 Similar to the implementation of `SOS_DEBUG`, we can define `SOS_WARNING` and disable some or all warnings. The implementation is almost identical to `SOS_DEBUG` and should be straightforward. The selection...
Workflow ``` input: for_each=dict(i=range(5)), concurrent=True python: expand='${ }' import time import os for rep in range(100): print(f'I am {rep} of task ${i} from {os.getpid()}') time.sleep(1) ``` Submit `Ctrl-C` after the...
Here is how currently benchmark workflows are automatically generated: https://github.com/gaow/random-nbs/blob/master/slides/20191201_DSC_SoS/sos/test_run.sos The two key elements in the design are: 1. Pre-specified input and output file names and `group_by` 2. Dependency by...
Here is a MWE. I have a script called `test.sos`: ``` [1] output: CONFIG['input'] run: touch ${output} [2] input: CONFIG['input'] output: CONFIG['output'] task: workdir = './', concurrent = True run:...
I got an error message that comes from this line of code: https://github.com/vatlab/sos/blob/cf3717e280d01a3ce5f6650f275aa1f91832d97a/src/sos/workers.py#L689 I wonder if it is a critical or benign error -- I am thinking a warning might...
Right not tasks status remains at `pending` if it is failed to submit.
As far as I remember, one of the advantages of using `mpiexec` to start commands on nodes is that the cluster system would be able to know the resources used...