TACC
Separator can cause cache syntax error
Describe the bug
If a module prepends (or appends, presumably) a path that ends with the separator character, it seems to cause a syntax error in the spider cache, which then causes a (non-fatal) error when running commands like module avail. Here is an example module command:
prepend_path("LD_LIBRARY_PATH", "/var/tmp/lib:")
Note the colon separator at the end of the path. The cache ends up with this entry for the module:
lpathA = {
= 1,
["/var/tmp/lib"] = 1,
},
Since this isn't valid syntax, running commands which interrogate the cache produce this error:
bin/luac: .cache/lmod/spiderT.bug_x86_64_Linux.lua:24: unexpected symbol near '='
The prepend_path statement obviously isn't very well constructed, but it can happen, so if Lmod could handle this scenario better it would be great. I ran into this because the Spack package manager was auto-generating modules with a statement like the above. I will also submit a PR to resolve that particular problem with Spack.
To Reproduce I've attached a simple reproducer, to be executed with:
$ env -i LMOD_ROOT=$LMOD_ROOT USER=$USER ./bug_report_template.sh
Desktop (please complete the following information):
- OS: Linux
- Linux distribution: SUSE
- Lmod Version: 8.7.37
$ module --miniConfig
Modules based on Lua: Version 8.7.37 2024-03-25 10:40 -06:00
by Robert McLay [email protected]
Changes from Default Configuration
----------------------------------
Name Where Set Default Value
---- --------- ------- -----
LFS_VERSION D 1.6.3 1.8.0
LMOD_AVAIL_STYLE E <system> grouped:system
LMOD_CONFIG_DIR E /etc/lmod /glade/u/apps/spack-deployments/gust/23.09/envs/public/util
LMOD_MODULERC E /glade/u/apps/gust/23.09/spack/opt/spack/lmod/8.7.37/gcc/7.5.0/rohu/lmod/etc/rc /glade/u/apps/spack-deployments/gust/23.09/envs/public/util/hidden-modules
LMOD_PACKAGE_PATH D nil /glade/u/apps/spack-deployments/gust/23.09/envs/public/util
LMOD_PAGER C less /usr/bin/less
LMOD_SITEPACKAGE_LOCATION Other /glade/u/apps/gust/23.09/spack/opt/spack/lmod/8.7.37/gcc/7.5.0/rohu/lmod/8.7.37/libexec/SitePackage.lua /glade/u/apps/spack-deployments/gust/23.09/envs/public/util/SitePackage.lua
LMOD_SYSTEM_DEFAULT_MODULES E __unknown__ ncarenv/23.09:craype/2.7.23:intel/2024.0.2:ncarcompilers/1.0.0:cray-mpich/8.1.27:netcdf/4.9.2
LMOD_SYSTEM_NAME E false gust
LMOD_TCLSH C tclsh /glade/u/apps/gust/23.09/spack/opt/spack/tcl/8.6.12/gcc/7.5.0/katj/bin/tclsh
MODULEPATH_ROOT E /glade/u/apps/gust/modules
PATH_TO_LUA C lua /glade/u/apps/gust/23.09/spack/opt/spack/lua/5.3.6/gcc/7.5.0/kflf/bin/lua
Where Set -> D: default, E: environment, C: configuration
lmod_cfg: lmod_config.lua SitePkg: SitePackage StdPkg: StandardPackage
Other: Set somewhere outside of normal locations
Thanks for the bug report with example test. This helps a great deal. Some packages do something special with an trailing delimiter. LUA env. vars do. So does tex/latex TEXINPUTS. So this cleaning up of trailing delimiters can not be done for all variables. So Lmod now removes extra delimiters for just PATH, LD_LIBRARY_PATH and MODULEPATH.
Please try the testing branch to see if it works for you. Thanks!
Apologies @rtmclay... I just realized I never got back to you after testing this! The change works great. Can LIBRARY_PATH be added to the list of vars to clean up? Other than that, it'd be great to get this into the next Lmod release.
Sure. cleaning up LIBRARY_PATH is now added and will be part of the next release.