Native installation problem

[IvanGrigorik]

I tried to install "pykokkos" using following native installation instructions. However, I had several problems, the last of which I did not solve:

1. The first problem is that these instructions and the requirements.txt file do not specify that nvcc is required. I understand that pykokkos-base can be built without nvcc at all. However, since it is specified in the build flag in a native installation, it is worth noting the need to have nvcc BEFORE running this instruction in a native installation: python setup.py install -- -DENABLE_LAYOUTS= ON -DENABLE_MEMORY_TRAITS=OFF -DENABLE_VIEW_RANKS=3 -DENABLE_CUDA=ON - DENABLE_THREADS=OFF -DENABLE_OPENMP=ON

2. The second problem is that the project is not built at the cmake build stage and stops with the following error:

-- Build files have been written to: /home/sifi/utils/pykokkos-base/_skbuild/linux-x86_64-3.12/cmake-build
[  1%] Building CXX object external/kokkos/core/src/CMakeFiles/kokkoscore.dir/impl/Kokkos_CPUDiscovery.cpp.o
/usr/include/c++/11/bits/std_function.h:435:145: error: parameter packs not expanded with ‘...’:
  435 |         function(_Functor&& __f)
      |                                                                                                                                                 ^
/usr/include/c++/11/bits/std_function.h:435:145: note:         ‘_ArgTypes’
/usr/include/c++/11/bits/std_function.h:530:146: error: parameter packs not expanded with ‘...’:
  530 |         operator=(_Functor&& __f)
      |                                                                                                                                                  ^
/usr/include/c++/11/bits/std_function.h:530:146: note:         ‘_ArgTypes’
gmake[2]: *** [external/kokkos/core/src/CMakeFiles/kokkoscore.dir/build.make:76: external/kokkos/core/src/CMakeFiles/kokkoscore.dir/impl/Kokkos_CPUDiscovery.cpp.o] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:1113: external/kokkos/core/src/CMakeFiles/kokkoscore.dir/all] Error 2
gmake: *** [Makefile:136: all] Error 2
Traceback (most recent call last):
  File "/home/sifi/utils/anaconda3/envs/pyk/lib/python3.12/site-packages/skbuild/setuptools_wrap.py", line 645, in setup
    cmkr.make(make_args, install_target=cmake_install_target, env=env)
  File "/home/sifi/utils/anaconda3/envs/pyk/lib/python3.12/site-packages/skbuild/cmaker.py", line 680, in make
    self.make_impl(clargs=clargs, config=config, source_dir=source_dir, install_target=install_target, env=env)
  File "/home/sifi/utils/anaconda3/envs/pyk/lib/python3.12/site-packages/skbuild/cmaker.py", line 704, in make_impl
    raise SKBuildError(

An error occurred while building with CMake.
  Command:
    cmake --build . --target install --config Release --
  Install target:
    install
  Source directory:
    /home/sifi/utils/pykokkos-base
  Working directory:
    /home/sifi/utils/pykokkos-base/_skbuild/linux-x86_64-3.12/cmake-build
Please check the install target is valid and see CMake's output for more information.

There are also several CMake warnings raised just before the error described above:

CMake Warning:
  Manually-specified variables were not used by the project:

    PYTHON_NumPy_INCLUDE_DIRS
    Python3_NumPy_INCLUDE_DIRS
    Python_EXECUTABLE
    Python_INCLUDE_DIR
    Python_LIBRARY
    Python_NumPy_INCLUDE_DIRS

[NaderAlAwar]

The first problem is that these instructions and the requirements.txt file do not specify that nvcc is required.

You are right in that the documentation does not specify that nvcc is required. When pykokkos-base is built with the -DENABLE_CUDA=ON flag, it will automatically use nvcc and it will fail if nvcc is not detected. You can build pykokkos-base without nvcc at all by either not specifying this flag or setting -DENABLE_CUDA=OFF (it might also be necessary to pass -DKokkos_ENABLE_CUDA=OFF).

The second problem is that the project is not built at the cmake build stage and stops with the following error:

This error is most likely an issue with the compiler version (https://github.com/NVlabs/instant-ngp/issues/119 is relevant). What are your nvcc and gcc versions?

There are also several CMake warnings raised just before the error described above:

These warnings should not affect the build process. Could you also share the CMake output prior to these warnings?

[IvanGrigorik]

I changed gcc/g++ to 10.5.0 and this problem went away.

However, I got another issue:

After building the project, the installation phase is started, and I got the following problem:

package init file 'kokkos/__init__.py' not found (or not a regular file)


running install
/usr/lib/python3/dist-packages/setuptools/command/easy_install.py:158: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
error: can't create or remove files in install directory

The following error occurred while trying to add or remove files in the
installation directory:

    [Errno 13] Permission denied: '/usr/local/lib/python3.10/dist-packages/test-easy-install-10915.write-test'

The installation directory you specified (via --install-dir, --prefix, or
the distutils default setting) was:

    /usr/local/lib/python3.10/dist-packages/

Perhaps your account does not have write access to this directory?  If the
installation directory is a system-owned directory, you may need to sign in
as the administrator or "root" account.  If you do not have administrative
access to this machine, you may wish to choose a different installation
directory, preferably one that is listed in your PYTHONPATH environment
variable.

For information on other options, you may wish to consult the
documentation at:

  https://setuptools.pypa.io/en/latest/deprecated/easy_install.html

Please make the appropriate changes for your system and try again.

When I try to do this through sudo, I get the following problems, which is logical since I did not do all the steps before as a superuser:

Traceback (most recent call last):
  File "/home/sifi/utils/pykokkos-base/setup.py", line 8, in <module>
    from skbuild import setup
ModuleNotFoundError: No module named 'skbuild'

Should I try to do the whole installation process as a root?

[NaderAlAwar]

[Errno 13] Permission denied: '/usr/local/lib/python3.10/dist-packages/test-easy-install-10915.write-test'

Seems like you're using the system python instead of one in a conda environment. Is this intentional? If not, on some machines I ended up having to use python3 instead of just python as that seems to point to the python in the environment. It is also possible that your PATH environment variable is not configured correctly. Can you try running which python just to be sure?

If you do intend on using the system python, I would imagine that you would have to use sudo to install packages. I have not done that before myself but I can help you with that. In this case, you would probably have to redo the whole installation as root.

[gliga]

Closing due to inactivity.