Cannot load module containing `execute`

[ktaletsk]

I am trying to load a module I created that registers Jupyter kernel from external conda environment. Here is the modulefile:

help([[Conda environment with Python Data Science packages
]])
execute{cmd="/opt/conda-envs/python-datascience/bin/python -m ipykernel install --user --name=Python-Data-Science", modeA={"load"}}
execute{cmd="jupyter kernelspec remove -y python-data-science", modeA={"unload"}}

Loading and unloading works perfectly from command line (browser refresh is required afterwards for kernel to show up). However, loading from jupyter-lmod does not work. After clicking load, the module shows up in the list of loaded modules, however checking from the command line does not confirm that it was loaded. Also, running load command takes few seconds, but in the jupyter-lmod loading is instant.

I started researching the issue and trying to understand how the extension works. I tried loading the lmod in Python and calling methods this way:

import lmod
await lmod.load("python-data-science")

This did not work either. I don't fully understand how does underlining the Python intefrace to lmod works and what the outputs mean. If I run lmod python --terse load python-data-science, the output is:

import os
os.environ["__LMOD_REF_COUNT_LOADEDMODULES"] = "lmod:1;python-data-science/0.1.0:1";
os.environ["LOADEDMODULES"] = "lmod:python-data-science/0.1.0";
os.environ["MODULEPATH"] = "/opt/apps/modulefiles/Linux:/opt/apps/modulefiles/Core:/opt/apps/lmod/lmod/modulefiles/Core";
os.environ["__LMOD_REF_COUNT__LMFILES_"] = "/opt/apps/lmod/lmod/modulefiles/Core/lmod.lua:1;/opt/apps/modulefiles/Linux/python-data-science/0.1.0.lua:1";
os.environ["_LMFILES_"] = "/opt/apps/lmod/lmod/modulefiles/Core/lmod.lua:/opt/apps/modulefiles/Linux/python-data-science/0.1.0.lua";
os.environ["_ModuleTable001_"] = "X01vZHVsZVRhYmxlXyA9IHsKTVR2ZXJzaW9uID0gMywKY19yZWJ1aWxkVGltZSA9IDcyMDAuMCwKY19zaG9ydFRpbWUgPSAwLjAwMTc3MjE2NTI5ODQ2MTksCmRlcHRoVCA9IHt9LApmYW1pbHkgPSB7fSwKbVQgPSB7Cmxtb2QgPSB7CmZuID0gIi9vcHQvYXBwcy9sbW9kL2xtb2QvbW9kdWxlZmlsZXMvQ29yZS9sbW9kLmx1YSIsCmZ1bGxOYW1lID0gImxtb2QiLApsb2FkT3JkZXIgPSAxLApwcm9wVCA9IHt9LApzdGFja0RlcHRoID0gMCwKc3RhdHVzID0gImFjdGl2ZSIsCnVzZXJOYW1lID0gImxtb2QiLAp3ViA9ICJNLip6ZmluYWwiLAp9LApbInB5dGhvbi1kYXRhLXNjaWVuY2UiXSA9IHsKZm4gPSAiL29wdC9hcHBzL21vZHVsZWZpbGVzL0xpbnV4L3B5dGhvbi1kYXRhLXNj";
os.environ["_ModuleTable002_"] = "aWVuY2UvMC4xLjAubHVhIiwKZnVsbE5hbWUgPSAicHl0aG9uLWRhdGEtc2NpZW5jZS8wLjEuMCIsCmxvYWRPcmRlciA9IDIsCnByb3BUID0ge30sCnN0YWNrRGVwdGggPSAwLApzdGF0dXMgPSAiYWN0aXZlIiwKdXNlck5hbWUgPSAicHl0aG9uLWRhdGEtc2NpZW5jZSIsCndWID0gIjAwMDAwMDAwMC4wMDAwMDAwMDEuKnpmaW5hbCIsCn0sCn0sCm1wYXRoQSA9IHsKIi9vcHQvYXBwcy9tb2R1bGVmaWxlcy9MaW51eCIsICIvb3B0L2FwcHMvbW9kdWxlZmlsZXMvQ29yZSIsICIvb3B0L2FwcHMvbG1vZC9sbW9kL21vZHVsZWZpbGVzL0NvcmUiLAp9LApzeXN0ZW1CYXNlTVBBVEggPSAiL29wdC9hcHBzL21vZHVsZWZpbGVzL0xpbnV4Oi9vcHQvYXBwcy9tb2R1bGVmaWxlcy9Db3Jl";
os.environ["_ModuleTable003_"] = "Oi9vcHQvYXBwcy9sbW9kL2xtb2QvbW9kdWxlZmlsZXMvQ29yZSIsCn0K";
os.environ["_ModuleTable_Sz_"] = "3";

All this command do is to modify the env variables, I don't see any commands that would trigger the script execution.

I am running jupyter-lmod 2.0.3 and JupyterLab 3.2.5

[cmd-ntrf]

Sorry for the late reply, this issue flew completely under my radar.

As you found out, jupyter-lmod executes Python code provided by lmod to maintain the environment variable. The issue with cmd() falls therefore on Lmod as it does not provide the Python code that would execute the command.

That said, if the only use case you have for cmd is to handle kernels, I might have a solution without cmd.