smartdispatch
smartdispatch copied to clipboard
Don't export the user's environment to the job
Fixes #157.
Currently the user's entire environment is exported. This is bad
practice because it makes jobs hard to reproduce and debug, and in some
cases it can even cause errors. The only variable that needs to be
exported is PBS_FILENAME
, which is used by the auto-resume script.
Coverage remained the same at 94.532% when pulling 2042df349319eb0ebff3ba8e3dda0e5a35d4723b on bartvm:master into c94f08e828d6deec453c054fd340e694ec0e0b9a on SMART-Lab:master.
@mgermain Test away.
@mgermain Ping.
@bartvm working on one cluster, not working on the other. Haven't had the time to investigate why yet.
Do you get an error? Or does it just not restart?
Without the -V on Mammouth some part of the environment is not loaded.
That's the entire point of the flag, no? But smart-dispatch shouldn't rely on the environment as set outside of the job. What environment variable does it try to access?
No, the point of the flag is to not copy the current environment. Your system still has to load the environment of the user in the job. It does on Helios and it does it weirdly and partially on mp2.
It does not try to access anything, I just saw that some modules are missing when -V is used. I have to investigate why.
That sounds weird... -V
shouldn't affect at all how the job's environment is set up beyond not importing environment variables from the launch environment. It's possible that Mammouth doesn't load .bashrc
at the beginning of a job. But in that case it's better to source .bashrc
explicitly in the generated PBS file, instead of relying on exporting with -V
IMO.