add support for Lmod-specific module family and module properties features

[boegel]

• --module-family command line option to include family statement
• --module-properties command line option to include property statements (note: requires that the properties are listed in $LMODRC file, see e.g. https://github.com/TACC/Lmod/blob/master/init/lmodrc.lua)

both options work with Tcl and Lua module file syntax

Example (Tcl):

export LMODRC=$PWD/lmodrc.lua
export EASYBUILD_MODULE_PROPERTIES=lmod:sticky
eb bzip2-1.0.6.eb -f --module-family test --module-properties=state:testing,arch:mic:offload

#%Module
...
if { [ string match "*tcl2lua.tcl" $env(_) ] } {
    family "test"
}
if { [ string match "*tcl2lua.tcl" $env(_) ] } {
    add-property "lmod" "sticky"
    add-property "state" "testing"
    add-property "arch" "mic:offload"
}
...

Example (Lua):

export LMODRC=$PWD/lmodrc.lua
export EASYBUILD_MODULE_PROPERTIES=lmod:sticky
export EASYBUILD_MODULE_SYNTAX=Lua
eb bzip2-1.0.6.eb -f --module-family test --module-properties=state:testing,arch:mic:offload

...
family("test")
add_property("lmod", "sticky")
add_property("state", "testing")
add_property("arch", "mic:offload")
...

@rtmclay: any comments?

[rtmclay]

The modulefiles for both TCL and Lua look good to me.

[hpcugentbot]

Refer to this link for build results (access rights to CI server needed): https://jenkins1.ugent.be/job/easybuild-framework-pr-builder/1606/ Test FAILed.

[hpcugentbot]

Refer to this link for build results (access rights to CI server needed): https://jenkins1.ugent.be/job/easybuild-framework-pr-builder/1607/ Test FAILed.

[hpcugentbot]

Refer to this link for build results (access rights to CI server needed): https://jenkins1.ugent.be/job/easybuild-framework-pr-builder/1608/ Test PASSed.

[boegel]

feature request by @pescobar for this: also add module_family and module_properties easyconfig parameters

[pescobar]

having the possibility to include module_family and module_properties in the easyconfigs would be nice to be able to rebuild a complete software stack in an automated way. Having the option only as a cli flag doesn´t allow an automated rebuild.

Also, I think it would be nice to define a list of Lmod properties which are supported by easybuild the same way we have now a list of supported moduleclasses. Maybe the properties included in the default file provided with Lmod https://github.com/TACC/Lmod/blob/master/init/lmodrc.lua is enough? or should be add few others? any suggestion on this?

[hpcugentbot]

Refer to this link for build results (access rights to CI server needed): https://jenkins1.ugent.be/job/easybuild-framework-pr-builder/1993/ EasyBuild framework unit test suite FAILed.

See https://jenkins1.ugent.be/job/easybuild-framework-pr-builder/1993/console for more details.

Please fix the reported issues by pushing additional commits to the branch corresponding with this pull request; contact @boegel if you're not sure what to do.

[geimer]

Regarding families: Instead of guarding the family statement in Tcl, the corresponding functionality could also be emulated using environment variables:

  switch [module-info mode] {
    load {
      if [info exists env(EB_FAMILY_FOO)] {
        puts "Conflicting module of 'foo' family detected!"
        puts "Please unload module \"$env(EB_FAMILY_FOO)\" first."
        exit
      } else {
        setenv EB_FAMILY_FOO "my_pkg"
      }
    }
  }

Not sure how much effort to put into it, though.

[boegel]

@geimer: that's interesting... I'd love to hear @rtmclay's thoughts on that.

[geimer]

@boegel It's a poor man's approach to prevent loading two modules of the same family. I've used it in the past and it works pretty well. Lmod is actually going beyond that by doing a switch. However, exiting with an error is probably the best one can do for Tmod as it doesn't automatically reload modules.

[fgeorgatos]

IMHO, exiting should be the default, in the name of safety, with the automated approach being a tunable (fi. we might have even distinct policies about what kind of swap policy kicks in, upon module swaps)

Also, I had a related idea in the past based on the concept of a var holding: FAMILY1:FAMILY2:FAMILY3 etc If we need to store individual software features though, then that gets more interesting..

On 14 November 2015 at 22:04, geimer [email protected] wrote:

@boegel https://github.com/boegel It's a poor man's approach to prevent loading two modules of the same family. I've used it in the past and it works pretty well. Lmod is actually going beyond that by doing a switch. However, exiting with an error is probably the best one can do for Tmod as it doesn't automatically reload modules.

— Reply to this email directly or view it on GitHub https://github.com/hpcugent/easybuild-framework/pull/1257#issuecomment-156749280 .

echo "sysadmin know better bash than english"|sed s/min/mins/
| sed 's/better bash/bash better/' # signal detected in a CERN forum

[hpcugentbot]

EasyBuild framework unit test suite FAILed.

See https://jenkins1.ugent.be/job/easybuild-framework-pr-builder/2497/console for more details.

Please fix the reported issues by pushing additional commits to the branch corresponding with this pull request; contact @boegel if you're not sure what to do.

[hpcugentbot]

EasyBuild framework unit test suite FAILed.

See https://jenkins1.ugent.be/job/easybuild-framework-pr-builder/2503/console for more details.

Please fix the reported issues by pushing additional commits to the branch corresponding with this pull request; contact @boegel if you're not sure what to do.

[JackPerdue]

This might not be where to request this (given the age of the issue) but I'd like to have an option to simply add another Lmod "conflict" statement to the generated module. In the case of Anaconda, I would want it to conflict with both Anaconda (other versions) and Python (any version). I don't want to do this for all builds, just select packages. If there was a "additional_conflicts" tag I could add to an EasyConfig, that would probably best (and simplest) for now.

[akesandgren]

@boegel Status?

[boegel]

Someone interested in this should pick up on this, but it's likely most of the effort here will have to be re-done due to all the merge conflicts...

I'll leave this open as a reminder, this is probably a feasible thing to work on during a hackathon/user meeting.

[boegel]

W.r.t. @JackPerdue's question: let's open a separate issue on that, since that's a feature request orthogonal to the "family" & "properties" concepts.