Installation issue: openmm python wrapper

[zzzoom]

Steps to reproduce the issue

$ spack spec -I openmm %[email protected] ~cuda
Input spec
--------------------------------
 -   openmm%[email protected]~cuda

Concretized
--------------------------------
 -   [email protected]%[email protected]~cuda~ipo build_system=cmake build_type=Release generator=make arch=linux-ubuntu24.04-zen4
[e]      ^[email protected]%[email protected]~doc+ncurses+ownlibs build_system=generic build_type=Release arch=linux-ubuntu24.04-zen4
[+]      ^[email protected]%[email protected]~graphviz~ipo~mscgen build_system=cmake build_type=Release generator=make arch=linux-ubuntu24.04-zen4
[e]          ^[email protected]%[email protected]~color build_system=autotools arch=linux-ubuntu24.04-zen4
[e]          ^[email protected]%[email protected]~doc+ncurses+ownlibs build_system=generic build_type=Release arch=linux-ubuntu24.04-zen4
[e]          ^[email protected]%[email protected]+lex~nls build_system=autotools patches=f8b85a0 arch=linux-ubuntu24.04-zen4
[e]          ^[email protected]%[email protected]~guile build_system=generic patches=599f134 arch=linux-ubuntu24.04-zen4
[+]          ^[email protected]%[email protected]+bz2+crypt+ctypes+dbm~debug+libxml2+lzma~nis~optimizations+pic+pyexpat+pythoncmd+readline+shared+sqlite3+ssl~tkinter+uuid+zlib build_system=generic patches=13fa8bf,b0615b2,ebdca64,f2fd060 arch=linux-ubuntu24.04-zen4
[e]              ^[email protected]%[email protected]+bzip2+curses+git~libunistring+libxml2+pic+shared+tar+xz build_system=autotools arch=linux-ubuntu24.04-zen4
[e]              ^[email protected]%[email protected]~docs+shared build_system=generic certs=mozilla arch=linux-ubuntu24.04-zen4
[+]      ^[email protected]%[email protected] build_system=generic arch=linux-ubuntu24.04-zen4
[e]      ^[email protected]%[email protected] build_system=autotools arch=linux-ubuntu24.04-zen4
[e]      ^[email protected]%[email protected]~guile build_system=generic patches=599f134 arch=linux-ubuntu24.04-zen4
[+]      ^[email protected]%[email protected] build_system=python_pip arch=linux-ubuntu24.04-zen4
[+]          ^[email protected]%[email protected] build_system=generic arch=linux-ubuntu24.04-zen4
[+]          ^[email protected]%[email protected] build_system=generic arch=linux-ubuntu24.04-zen4
[+]          ^[email protected]%[email protected] build_system=generic arch=linux-ubuntu24.04-zen4
[+]      ^[email protected]%[email protected] build_system=python_pip patches=873745d arch=linux-ubuntu24.04-zen4
[+]          ^[email protected]%[email protected]~bignuma~consistent_fpcsr+dynamic_dispatch+fortran~ilp64+locking+pic+shared build_system=makefile symbol_suffix=none threads=none arch=linux-ubuntu24.04-zen4
[e]              ^[email protected]%[email protected]~cpanm+opcode+open+shared+threads build_system=generic arch=linux-ubuntu24.04-zen4
[+]          ^[email protected]%[email protected] build_system=python_pip arch=linux-ubuntu24.04-zen4
[e]              ^[email protected]%[email protected] build_system=python_pip patches=0f0b1bd arch=linux-ubuntu24.04-zen4
[+]              ^[email protected]%[email protected] build_system=python_pip arch=linux-ubuntu24.04-zen4
[+]                  ^[email protected]%[email protected] build_system=python_pip arch=linux-ubuntu24.04-zen4
[+]                      ^[email protected]%[email protected] build_system=python_pip arch=linux-ubuntu24.04-zen4
[+]      ^[email protected]%[email protected]+bz2+crypt+ctypes+dbm~debug+libxml2+lzma~nis~optimizations+pic+pyexpat+pythoncmd+readline+shared+sqlite3+ssl~tkinter+uuid+zlib build_system=generic patches=13fa8bf,b0615b2,ebdca64,f2fd060 arch=linux-ubuntu24.04-zen4
[+]          ^[email protected]%[email protected]~debug~pic+shared build_system=generic arch=linux-ubuntu24.04-zen4
[e]              ^[email protected]%[email protected] build_system=autotools arch=linux-ubuntu24.04-zen4
[+]          ^[email protected]%[email protected]+libbsd build_system=autotools arch=linux-ubuntu24.04-zen4
[+]              ^[email protected]%[email protected] build_system=autotools arch=linux-ubuntu24.04-zen4
[+]                  ^[email protected]%[email protected] build_system=autotools arch=linux-ubuntu24.04-zen4
[+]          ^[email protected]%[email protected] build_system=autotools arch=linux-ubuntu24.04-zen4
[e]          ^[email protected]%[email protected]+bzip2+curses+git~libunistring+libxml2+pic+shared+tar+xz build_system=autotools arch=linux-ubuntu24.04-zen4
[+]          ^[email protected]%[email protected] build_system=autotools arch=linux-ubuntu24.04-zen4
[+]          ^[email protected]%[email protected]~obsolete_api build_system=autotools patches=4885da3 arch=linux-ubuntu24.04-zen4
[+]          ^[email protected]%[email protected]~symlinks+termlib abi=none build_system=autotools patches=7a351bc arch=linux-ubuntu24.04-zen4
[e]          ^[email protected]%[email protected]~docs+shared build_system=generic certs=mozilla arch=linux-ubuntu24.04-zen4
[e]          ^[email protected]%[email protected] build_system=autotools arch=linux-ubuntu24.04-zen4
[+]          ^[email protected]%[email protected] build_system=autotools patches=bbf97f1 arch=linux-ubuntu24.04-zen4
[+]          ^[email protected]%[email protected]+column_metadata+dynamic_extensions+fts~functions+rtree build_system=autotools arch=linux-ubuntu24.04-zen4
[+]          ^[email protected]%[email protected] build_system=autotools arch=linux-ubuntu24.04-zen4
[+]          ^[email protected]%[email protected]~pic build_system=autotools libs=shared,static arch=linux-ubuntu24.04-zen4
[+]          ^[email protected]%[email protected]+compat+new_strategies+opt+pic+shared build_system=autotools arch=linux-ubuntu24.04-zen4
[+]      ^[email protected]%[email protected] build_system=generic arch=linux-ubuntu24.04-zen4
[+]      ^[email protected]%[email protected] build_system=autotools arch=linux-ubuntu24.04-zen4
[+]          ^[email protected]%[email protected]~jit+multibyte build_system=autotools arch=linux-ubuntu24.04-zen4
[e]          ^[email protected]%[email protected] build_system=autotools arch=linux-ubuntu24.04-zen4

Error message

Spack sets openmm's PYTHONPATH properly, but its python wrapper egg is extracted to a subdirectory inside site-packages so importing the module fails.

Error message

$ spack load [email protected] ~cuda
$ spack load [email protected]
$ echo $PYTHONPATH | tr ':' '\n'
/home/bc/repo/spack/opt/spack/linux-ubuntu24.04-zen4/gcc-13.2.0/openmm-8.1.1-kmn2rzqdzp6gspku2f46nkw6q4w4t7sb/lib/python3.11/site-packages
/home/bc/repo/spack/opt/spack/linux-ubuntu24.04-zen4/gcc-13.2.0/py-numpy-1.26.4-tpsazw4xuxnboj6ssj5ddcy3bdxdh6cb/lib/python3.11/site-packages
/home/bc/repo/spack/opt/spack/linux-ubuntu24.04-zen4/gcc-13.2.0/python-venv-1.0-3ppprpw2bw7evapokawatal2w7m6cmlu/lib/python3.11/site-packages
$ python `spack location -i [email protected] %[email protected] ~cuda`/examples/benchmark.py
Traceback (most recent call last):
  File "/home/bc/repo/spack/opt/spack/linux-ubuntu24.04-zen4/gcc-13.2.0/openmm-8.1.1-kmn2rzqdzp6gspku2f46nkw6q4w4t7sb/examples/benchmark.py", line 2, in 
    import openmm.app as app
ModuleNotFoundError: No module named 'openmm'
$ ls -l /home/bc/repo/spack/opt/spack/linux-ubuntu24.04-zen4/gcc-13.2.0/openmm-8.1.1-kmn2rzqdzp6gspku2f46nkw6q4w4t7sb/lib/python3.11/site-packages
total 4
drwxr-sr-x 5 bc bc 4096 May 10 03:32 OpenMM-8.1.1-py3.11-linux-x86_64.egg
$

Information on your system

• Spack: 0.22.0.dev0 (636d479e5f3eaa35862d59f2b5148d30e9b59d42)
• Python: 3.12.3
• Platform: linux-ubuntu24.04-zen4
• Concretizer: clingo

Additional information

spack-build-out.txt spack-build-env.txt

General information

• [X] I have run spack debug report and reported the version of Spack/Python/Platform
• [X] I have run spack maintainers <name-of-the-package> and @mentioned any maintainers
• [X] I have uploaded the build log and environment files
• [X] I have searched the issues of this repo and believe this is not a duplicate

[zzzoom]

I'd give this a shot but someone familiar with python setuptools shenanigans should figure it out much faster than me.

[haampie]

Looks like a regression between 8.0.0 and 8.1.1.

Maybe easiest to find the issue with spack develop / git bisect

[zzzoom]

They switched from distutils to setuptools in wrappers/python/setup.py but I don't know how python setup.py install --prefix={self.prefix} results in an egg being created and then unpacked.

-from distutils.core import setup
+from setuptools import setup

[haampie]

Yeah, also bisected it to 76be194cbd09c7f595624984ff7f38c73451377a

[haampie]

https://github.com/openmm/openmm/issues/4538