SmartSim icon indicating copy to clipboard operation
SmartSim copied to clipboard

Load environment modules before launching an `Experiment`

Open b-fg opened this issue 2 years ago • 2 comments

Description

Specially in clusters, sometimes the environmental modules needed for the execution of an Experiment are incompatible with the modules needed for SmartSim and SmartRedis. It would be useful to be able to load modules in the experiment subshell before the run command is launched.

Justification

HPC cluster users, specially, would benefit from this feature.

Implementation Strategy

To be able to pass a bash script which is called before the Experiment.start method in the same subshell. The bash script may contain the environmental modules needed for the executable.

b-fg avatar Sep 17 '23 18:09 b-fg

Because of compatibility issues, I am currently using SmartSim v0.4.2 in the cluster. It would be appreciated if someone can point me to a quick fix or a way to do this in that version too. Thanks!

b-fg avatar Sep 17 '23 18:09 b-fg

Regarding SmartSim v0.4.2, I managed to get this working by passing batch_settings when creating a model and use the add_preamble method to source the relevant bash file that contains the modules. The downside of this of course is that everytime a model is restarted a new job needs to be allocated, and this can slowdown the execution. Ideally, I would like to find a method to load the modules in the sub-shell of the model run command, so I only need to submit a single job with the necessary resources.

b-fg avatar Sep 19 '23 16:09 b-fg