signac-flow
signac-flow copied to clipboard
Support modification of the subprocess environment created when groups fork to execute operations
Feature description
Operations that require forking, either via an explicit fork
directive or using any other directives that trigger forking like nranks
(via MPI) or executable
, end up executing operations in a subprocess of the process where the job script and the original signac-flow launch occur. Our Jinja templates offer various methods for modifying the parent process, and users can also write custom templates to modify the environment however they need. However, we don't currently offer any method for controlling the child processes. It would be nice to include some way of doing so.
Proposed solution
Currently the best solution I was able to suggest was to use the executable
directive to just include a compound command, something like DO_THIS && DO_THIS && NOW_RUN_ORIGINAL_COMMAND
. Maybe something like this is the best that we can do; if so we should document it as such in places like here.
We could allow for a setup script for any forked commands to ensure that the modules are loaded as necessary. Otherwise we could document using the correct configuration files on a user shell to load necessary modules.
This could be like a template effectively except there would be no default.
Edit: Elaborated.