scikit-build-core
scikit-build-core copied to clipboard
scikit-build
Difficulty Setting up RPATH with scikit-build Core
I am facing some difficulty while setting up RPATH. In the past, I was using ${CMAKE_INSTALL_PREFIX}/lib for setting up RPATH. However, since I have moved to scikit-build core, I have set this to ${SKBUILD_PLATLIB_DIR}/${install-dir}/lib, but it seems the path is set as /tmp/tmp4iut2668/wheel/platlib/${install-dir}/lib and I don't know if this is the correct way.
Below is how I previously configured RPATH:
# Setup the RPATH correctly
macro(pyne_configure_rpath)
# use, i.e. don't skip the full RPATH for the build tree
SET(CMAKE_SKIP_BUILD_RPATH FALSE)
# when building, don't use the install RPATH already
# (but later on when installing)
SET(CMAKE_BUILD_WITH_INSTALL_RPATH FALSE)
SET(CMAKE_INSTALL_RPATH "${CMAKE_INSTALL_PREFIX}/lib")
# add the automatically determined parts of the RPATH
# which point to directories outside the build tree to the install RPATH
SET(CMAKE_INSTALL_RPATH_USE_LINK_PATH TRUE)
# the RPATH to be used when installing, but only if it's not a system directory
LIST(FIND CMAKE_PLATFORM_IMPLICIT_LINK_DIRECTORIES
"${CMAKE_INSTALL_PREFIX}/lib" isSystemDir)
IF("${isSystemDir}" STREQUAL "-1")
SET(CMAKE_INSTALL_RPATH "${CMAKE_INSTALL_PREFIX}/lib")
GET_FILENAME_COMPONENT(cxxCompilerRoot ${CMAKE_CXX_COMPILER} DIRECTORY)
GET_FILENAME_COMPONENT(cxxCompilerRoot ${cxxCompilerRoot} DIRECTORY)
IF(NOT "${CMAKE_INSTALL_RPATH}" STREQUAL "${cxxCompilerRoot}")
SET(CMAKE_INSTALL_RPATH "${CMAKE_INSTALL_RPATH}:${cxxCompilerRoot}/lib")
ENDIF (NOT "${CMAKE_INSTALL_RPATH}" STREQUAL "${cxxCompilerRoot}")
ENDIF("${isSystemDir}" STREQUAL "-1")
MESSAGE("-- CMAKE_INSTALL_RPATH: ${CMAKE_INSTALL_RPATH}")
endmacro()
How about setting the RPATH relative to $ORIGIN. The issue here is that the installed artifacts have to be fully relocatable, i.e. if you copy the files around, it should still be executable, so hard-coding absolute path RPATH breaks this.
How you put RPATH depends a lot on how/where you wish to install all dependent libraries, do you expect to have system installed veraions of those libraries, where do you want to package and what is their standard. You have to ve careful not to hard-code most of these stuff, and instead move them to pyproject.toml when possible.
Thank you for your prompt response. You're correct. The Python module I'm using is build python3 -m build -w, typically generates the build in the temp folder.
Let me explain what I'm trying to achieve: I have a library named libpyne.so, which is installed by CMake to the site-packages/pyne/core/lib folder. My Cython modules are stored in site-packages/pyne/. Therefore, I'm seeking a method to set the relative path for libpyne.so so that other modules can easily locate it. Currently, I have to configure LD_LIBRARY_PATH in the terminal, but this isn't a robust approach.
Is there any configuration available in pyproject.toml for this? I've set my install directory to wheel.install-dir = "pyne/core" .
You can set INSTALL_RPATH and use ${ORIGIN} inside it to point to the appropriate paths. Iiuc the cython library is defined by add_library or python_add_library right?
Note that for windows is a bit trickier, but you can get around it by making sure the dll is in the same folder.
There are 2 more design approaches to define those libraries as static, or to load the libraries manually within the python module.
Ps. The 3 recent issues seem quite similar to each other. It would be nice if the issues can be gathered for documentation improvement
I am happy to update the documentation. There are two ways to proceed. One is to create a document page named “Migrating from Pure CMake to Skbuild,” and the other is to add an example project in the tests folder of the repository. Please let me know which procedure I should follow.
Here are my steps:
-
A
pyproject.tomlfile that contains cmake.install-dir. -
A
CMakeLists.txtfile that includes a library pointing toinstall-dir, and Cython and Fortran modules to a custom location that is linked with the library.
I am encountering difficulties linking f2py with my CPython module. I've tried two approaches, but I keep getting the same error:
Python 3.11.8 (main, Feb 7 2024, 21:52:08) [GCC 13.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from spatial_solvers import ahot_script
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/mnt/Data/Projects/pyne/tests/spatial_solvers/ahot_script.py", line 11, in <module>
import pyne.spatialsolver
File "/mnt/Data/Projects/pyne/.env/lib/python3.11/site-packages/pyne/spatialsolver.py", line 11, in <module>
import pyne.transport_spatial_methods as transport_spatial_methods
ImportError: /mnt/Data/Projects/pyne/.env/lib/python3.11/site-packages/pyne/transport_spatial_methods.cpython-311-x86_64-linux-gnu.so: undefined symbol: f2pywrapmain_
Method 1:
add_library(fortranobject OBJECT "${F2PY_INCLUDE_DIR}/fortranobject.c")
target_link_libraries(fortranobject PUBLIC Python::NumPy)
target_include_directories(fortranobject PUBLIC "${F2PY_INCLUDE_DIR}")
set_property(TARGET fortranobject PROPERTY POSITION_INDEPENDENT_CODE ON)
# Common variables
set(f2py_module_name "transport_spatial_methods")
set(fortran_src_file "${PROJECT_SOURCE_DIR}/src/transport_spatial_methods/3d/main.f90")
add_custom_command(
OUTPUT ${f2py_module_name}module.c ${f2py_module_name}-f2pywrappers.f
DEPENDS ${fortran_src_file}
VERBATIM
COMMAND "${Python_EXECUTABLE}" -m numpy.f2py
"${fortran_src_file}" -m "${f2py_module_name}" --lower)
python_add_library(${f2py_module_name} MODULE "${CMAKE_CURRENT_BINARY_DIR}/${f2py_module_name}module.c"
"${fortran_src_file}" WITH_SOABI)
target_link_libraries(${f2py_module_name} PRIVATE fortranobject ${CMAKE_PROJECT_NAME})
# Configure the RPATH
set_target_properties(${f2py_module_name} PROPERTIES
INSTALL_RPATH "$ORIGIN/core/lib:$ORIGIN/../numpy.libs:$ORIGIN/../numpy/f2py/src"
INSTALL_RPATH_USE_LINK_PATH TRUE
)
add_dependencies(${f2py_module_name} ${fortranobject})
install(TARGETS ${f2py_module_name} DESTINATION ../.) # wheel.install-dir=pyne/core wheel.packages=pyne
install(TARGETS fortranobject LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR})
Method 2:
include_directories(${F2PY_INCLUDE_DIR})
# Common variables
set(f2py_module_name "transport_spatial_methods")
set(fortran_src_file "${PROJECT_SOURCE_DIR}/src/transport_spatial_methods/3d/main.f90")
set(f2py_module_c "${f2py_module_name}module.c")
# Generate sources
add_custom_target(
genpyf
DEPENDS "${fortran_src_file}"
)
add_custom_command(
OUTPUT "${CMAKE_CURRENT_BINARY_DIR}/${f2py_module_c}"
COMMAND ${Python_EXECUTABLE} -m "numpy.f2py"
-I${PROJECT_BINARY_DIR}/src
"${fortran_src_file}"
-m "${f2py_module_name}"
--lower # Important
DEPENDS ${CMAKE_PROJECT_NAME}
)
# Set up target
python_add_library( ${f2py_module_name} MODULE WITH_SOABI
"${CMAKE_CURRENT_BINARY_DIR}/${f2py_module_c}" # Generated
"${F2PY_INCLUDE_DIR}/fortranobject.c" # From NumPy
"${fortran_src_file}" # Fortran source(s)
)
# Depend on sources
target_include_directories(${f2py_module_name} PUBLIC ${F2PY_INCLUDE_DIR})
target_link_libraries(${f2py_module_name} PUBLIC Python::NumPy ${LAPACK_LIBRARIES} ${BLAS_LIBRARIES} ${CMAKE_PROJECT_NAME})
# Configure the RPATH
set_target_properties(${f2py_module_name} PROPERTIES
INSTALL_RPATH "$ORIGIN/core/lib:$ORIGIN/../numpy.libs"
INSTALL_RPATH_USE_LINK_PATH TRUE
)
add_dependencies(${f2py_module_name} genpyf)
install(TARGETS ${f2py_module_name} DESTINATION ../.)
Can you run readelf -d on the transport_spatial_methods.cpython-311-x86_64-linux-gnu.so. The other issue I see is with the installation paths of the targets, but I'm not sure in which library that symbol is defined. Curious when you activate a venv, does it alter the LD_LIBRARY_PATH?
add an example project in the tests folder of the repository.
There are example projects already in the docs that are being tested. But they do not cover the intricacies of RPATH or windows library loading, so it can be improved in that aspect. I think the examples can be further split based on OBJECT, STATIC, SHARED library types for the core library.
readelf -d transport_spatial_methods.cpython-311-x86_64-linux-gnu.so
Dynamic section at offset 0x12a48 contains 29 entries:
Tag Type Name/Value
0x0000000000000001 (NEEDED) Shared library: [libpyne.so]
0x0000000000000001 (NEEDED) Shared library: [libgfortran.so.5]
0x0000000000000001 (NEEDED) Shared library: [libm.so.6]
0x0000000000000001 (NEEDED) Shared library: [libgcc_s.so.1]
0x0000000000000001 (NEEDED) Shared library: [libc.so.6]
0x000000000000001d (RUNPATH) Library runpath: [$ORIGIN/core/lib:$ORIGIN/../numpy.libs:$ORIGIN/../numpy/f2py/src]
0x000000000000000c (INIT) 0x5000
0x000000000000000d (FINI) 0xddd0
0x0000000000000019 (INIT_ARRAY) 0x13a38
0x000000000000001b (INIT_ARRAYSZ) 8 (bytes)
0x000000000000001a (FINI_ARRAY) 0x13a40
0x000000000000001c (FINI_ARRAYSZ) 8 (bytes)
0x000000006ffffef5 (GNU_HASH) 0x260
0x0000000000000005 (STRTAB) 0x1748
0x0000000000000006 (SYMTAB) 0x308
0x000000000000000a (STRSZ) 3949 (bytes)
0x000000000000000b (SYMENT) 24 (bytes)
0x0000000000000003 (PLTGOT) 0x13fe8
0x0000000000000002 (PLTRELSZ) 2160 (bytes)
0x0000000000000014 (PLTREL) RELA
0x0000000000000017 (JMPREL) 0x3810
0x0000000000000007 (RELA) 0x28c8
0x0000000000000008 (RELASZ) 3912 (bytes)
0x0000000000000009 (RELAENT) 24 (bytes)
0x000000006ffffffe (VERNEED) 0x2868
0x000000006fffffff (VERNEEDNUM) 2
0x000000006ffffff0 (VERSYM) 0x26b6
0x000000006ffffff9 (RELACOUNT) 48
0x0000000000000000 (NULL) 0x0
readelf -d libpyne.so
Dynamic section at offset 0x3685468 contains 32 entries:
Tag Type Name/Value
0x0000000000000001 (NEEDED) Shared library: [libhdf5_serial.so.103]
0x0000000000000001 (NEEDED) Shared library: [libm.so.6]
0x0000000000000001 (NEEDED) Shared library: [libblas.so.3]
0x0000000000000001 (NEEDED) Shared library: [liblapack.so.3]
0x0000000000000001 (NEEDED) Shared library: [libgfortran.so.5]
0x0000000000000001 (NEEDED) Shared library: [libstdc++.so.6]
0x0000000000000001 (NEEDED) Shared library: [libgcc_s.so.1]
0x0000000000000001 (NEEDED) Shared library: [libc.so.6]
0x000000000000000e (SONAME) Library soname: [libpyne.so]
0x000000000000000c (INIT) 0x45000
0x000000000000000d (FINI) 0x319da88
0x0000000000000019 (INIT_ARRAY) 0x3685ea8
0x000000000000001b (INIT_ARRAYSZ) 136 (bytes)
0x000000000000001a (FINI_ARRAY) 0x3685f30
0x000000000000001c (FINI_ARRAYSZ) 8 (bytes)
0x000000006ffffef5 (GNU_HASH) 0x260
0x0000000000000005 (STRTAB) 0xea98
0x0000000000000006 (SYMTAB) 0x3500
0x000000000000000a (STRSZ) 97356 (bytes)
0x000000000000000b (SYMENT) 24 (bytes)
0x0000000000000003 (PLTGOT) 0x3686fe8
0x0000000000000002 (PLTRELSZ) 24432 (bytes)
0x0000000000000014 (PLTREL) RELA
0x0000000000000017 (JMPREL) 0x3ec90
0x0000000000000007 (RELA) 0x27848
0x0000000000000008 (RELASZ) 95304 (bytes)
0x0000000000000009 (RELAENT) 24 (bytes)
0x000000006ffffffe (VERNEED) 0x27608
0x000000006fffffff (VERNEEDNUM) 6
0x000000006ffffff0 (VERSYM) 0x266e4
0x000000006ffffff9 (RELACOUNT) 3510
0x0000000000000000 (NULL) 0x0
I also tried to link Python:NumPy, F2PY_INCLUDE_DIR with libpyne.so but noticed it shows f2py inking issues.
Curious when you activate a venv, does it alter the
LD_LIBRARY_PATH?
I think no, because I had previously defined the LD_LIBRARY_PATH for the PyNE library in my shell. As a result, I was unable to identify this issue. However, upon resetting it, I discovered that the linking is not working.
I was worried the $ORIGIN was being expanded, but it seems to work fine. The other part that I can think of is where do the symbols f2pywrapmain_ get defined. I assume these are in the fortranobject or ${f2py_module_name}. If it were the latter you should consult with numpy.f2py to see if they install the library in a specific location.
What puzzles me is that the getting started example does not have any special handling, and you seem to be following the same approach in the first method. In the example there is no external library to link to, so it is more likely to work, but if f2pywrapmain_ is something defined from numpy.f2py, the same issue should happen there as well :thinking:
Can you run readelf -d in the build directory as well. Normally this linking issue would occur at build time as well if you didn't have appropriate link dependencies, but that did not seem to happen right?
Path : ~/temp/scikit-build/pyne/
readelf -d transport_spatial_methods.cpython-311-x86_64-linux-gnu.so
Dynamic section at offset 0x12a48 contains 29 entries:
Tag Type Name/Value
0x0000000000000001 (NEEDED) Shared library: [libpyne.so]
0x0000000000000001 (NEEDED) Shared library: [libgfortran.so.5]
0x0000000000000001 (NEEDED) Shared library: [libm.so.6]
0x0000000000000001 (NEEDED) Shared library: [libgcc_s.so.1]
0x0000000000000001 (NEEDED) Shared library: [libc.so.6]
0x000000000000001d (RUNPATH) Library runpath: [/home/tahmid/temp/scikit-build/src:]
0x000000000000000c (INIT) 0x5000
0x000000000000000d (FINI) 0xddc4
0x0000000000000019 (INIT_ARRAY) 0x13a38
0x000000000000001b (INIT_ARRAYSZ) 8 (bytes)
0x000000000000001a (FINI_ARRAY) 0x13a40
0x000000000000001c (FINI_ARRAYSZ) 8 (bytes)
0x000000006ffffef5 (GNU_HASH) 0x260
0x0000000000000005 (STRTAB) 0x1748
0x0000000000000006 (SYMTAB) 0x308
0x000000000000000a (STRSZ) 3920 (bytes)
0x000000000000000b (SYMENT) 24 (bytes)
0x0000000000000003 (PLTGOT) 0x13fe8
0x0000000000000002 (PLTRELSZ) 2160 (bytes)
0x0000000000000014 (PLTREL) RELA
0x0000000000000017 (JMPREL) 0x37f0
0x0000000000000007 (RELA) 0x28a8
0x0000000000000008 (RELASZ) 3912 (bytes)
0x0000000000000009 (RELAENT) 24 (bytes)
0x000000006ffffffe (VERNEED) 0x2848
0x000000006fffffff (VERNEEDNUM) 2
0x000000006ffffff0 (VERSYM) 0x2698
0x000000006ffffff9 (RELACOUNT) 48
0x0000000000000000 (NULL) 0x0
Path: ~/temp/scikit-build/src
readelf -d libpyne.so
Dynamic section at offset 0x3685468 contains 33 entries:
Tag Type Name/Value
0x0000000000000001 (NEEDED) Shared library: [libhdf5_serial.so.103]
0x0000000000000001 (NEEDED) Shared library: [libm.so.6]
0x0000000000000001 (NEEDED) Shared library: [libblas.so.3]
0x0000000000000001 (NEEDED) Shared library: [liblapack.so.3]
0x0000000000000001 (NEEDED) Shared library: [libgfortran.so.5]
0x0000000000000001 (NEEDED) Shared library: [libstdc++.so.6]
0x0000000000000001 (NEEDED) Shared library: [libgcc_s.so.1]
0x0000000000000001 (NEEDED) Shared library: [libc.so.6]
0x000000000000000e (SONAME) Library soname: [libpyne.so]
0x000000000000001d (RUNPATH) Library runpath: [/usr/lib/x86_64-linux-gnu/hdf5/serial:]
0x000000000000000c (INIT) 0x45000
0x000000000000000d (FINI) 0x319da88
0x0000000000000019 (INIT_ARRAY) 0x3685ea8
0x000000000000001b (INIT_ARRAYSZ) 136 (bytes)
0x000000000000001a (FINI_ARRAY) 0x3685f30
0x000000000000001c (FINI_ARRAYSZ) 8 (bytes)
0x000000006ffffef5 (GNU_HASH) 0x260
0x0000000000000005 (STRTAB) 0xea98
0x0000000000000006 (SYMTAB) 0x3500
0x000000000000000a (STRSZ) 97356 (bytes)
0x000000000000000b (SYMENT) 24 (bytes)
0x0000000000000003 (PLTGOT) 0x3686fe8
0x0000000000000002 (PLTRELSZ) 24432 (bytes)
0x0000000000000014 (PLTREL) RELA
0x0000000000000017 (JMPREL) 0x3ec90
0x0000000000000007 (RELA) 0x27848
0x0000000000000008 (RELASZ) 95304 (bytes)
0x0000000000000009 (RELAENT) 24 (bytes)
0x000000006ffffffe (VERNEED) 0x27608
0x000000006fffffff (VERNEEDNUM) 6
0x000000006ffffff0 (VERSYM) 0x266e4
0x000000006ffffff9 (RELACOUNT) 3510
0x0000000000000000 (NULL) 0x0
My setup
Initial setup at project/:
# Find Python and NumPy
find_package(
Python
COMPONENTS Interpreter Development.Module NumPy
REQUIRED
)
# Check if spatial solver is requested by user
if (BUILD_SPATIAL_SOLVER)
# Find f2py, if building spatial solver
# Grab the variables from a local Python installation F2PY headers
execute_process(
COMMAND "${Python_EXECUTABLE}" -c
"import numpy.f2py; print(numpy.f2py.get_include())"
OUTPUT_VARIABLE F2PY_INCLUDE_DIR
OUTPUT_STRIP_TRAILING_WHITESPACE
)
# Try to find LAPACK and BLAS
find_package(LAPACK)
endif()
Main Library at project/src:
set(TRANSPORT_SPATIAL_METHODS_SRCS
"transport_spatial_methods/3d/ahotn_kernel_module.f90"
"transport_spatial_methods/3d/angle.f90"
"transport_spatial_methods/3d/check.f90"
"transport_spatial_methods/3d/dgfem_kernel.f90"
"transport_spatial_methods/3d/echo.f90"
"transport_spatial_methods/3d/geompack.f90"
"transport_spatial_methods/3d/igeompack.f90"
"transport_spatial_methods/3d/inner.f90"
"transport_spatial_methods/3d/invar.f90"
"transport_spatial_methods/3d/output.f90"
"transport_spatial_methods/3d/output_phi.f90"
"transport_spatial_methods/3d/p.f90"
"transport_spatial_methods/3d/precision_module.f90"
"transport_spatial_methods/3d/read_inflow_ahotn.f90"
"transport_spatial_methods/3d/read_inflow_dgfem.f90"
"transport_spatial_methods/3d/read_inflow_sct_step.f90"
"transport_spatial_methods/3d/readsrc.f90"
"transport_spatial_methods/3d/readxs.f90"
"transport_spatial_methods/3d/sct_module.f90"
"transport_spatial_methods/3d/sct_step_kernel_module.f90"
"transport_spatial_methods/3d/solvar.f90"
"transport_spatial_methods/3d/solve.f90"
"transport_spatial_methods/3d/sweep_ahotn_l.f90"
"transport_spatial_methods/3d/sweep_ahotn_nefd.f90"
"transport_spatial_methods/3d/sweep_dgfem.f90"
"transport_spatial_methods/3d/sweep_sct_step.f90"
"transport_spatial_methods/3d/timevar.f90"
"transport_spatial_methods/3d/trackroutines.f90"
"transport_spatial_methods/3d/trackstruct.f90"
"transport_spatial_methods/3d/version.f90"
#"transport_spatial_methods/3d/main.f90"
)
if(BUILD_SPATIAL_SOLVER)
set(PYNE_SRCS "${PYNE_SRCS}" "${TRANSPORT_SPATIAL_METHODS_SRCS}")
endif()
add_library(${CMAKE_PROJECT_NAME} ${PYNE_SRCS})
if(BUILD_SPATIAL_SOLVER)
target_link_libraries(${CMAKE_PROJECT_NAME}
PRIVATE ${LIBS_HDF5} ${BLAS_LIBRARIES} ${LAPACK_LIBRARIES}
)
endif()
Fortarn module at project/pyne:
if(BUILD_SPATIAL_SOLVER)
include_directories(${F2PY_INCLUDE_DIR})
# Common variables
set(f2py_module_name "transport_spatial_methods")
set(fortran_src_file "${PROJECT_SOURCE_DIR}/src/transport_spatial_methods/3d/main.f90")
set(f2py_module_c "${f2py_module_name}module.c")
# Generate sources
add_custom_target(
genpyf
DEPENDS "${fortran_src_file}"
)
add_custom_command(
OUTPUT "${CMAKE_CURRENT_BINARY_DIR}/${f2py_module_c}"
COMMAND ${Python_EXECUTABLE} -m "numpy.f2py"
-I${PROJECT_BINARY_DIR}/src
"${fortran_src_file}"
-m "${f2py_module_name}"
--lower # Important
DEPENDS ${CMAKE_PROJECT_NAME}
)
# Set up target
python_add_library( ${f2py_module_name} MODULE WITH_SOABI
"${CMAKE_CURRENT_BINARY_DIR}/${f2py_module_c}" # Generated
"${F2PY_INCLUDE_DIR}/fortranobject.c" # From NumPy
"${fortran_src_file}" # Fortran source(s)
)
# Depend on sources
target_include_directories(${f2py_module_name} PUBLIC ${F2PY_INCLUDE_DIR})
target_link_libraries(${f2py_module_name} PUBLIC Python::NumPy ${LAPACK_LIBRARIES} ${BLAS_LIBRARIES} ${CMAKE_PROJECT_NAME})
# Configure the RPATH
set_target_properties(${f2py_module_name} PROPERTIES
INSTALL_RPATH "$ORIGIN/core/lib"
)
add_dependencies(${f2py_module_name} genpyf)
install(TARGETS ${f2py_module_name} DESTINATION ../.)
endif()
Nitpick: don't write stuff like *_SRCS, these are relics of autotools. Use target_source and such. Also avoid using ${PROJECT_NAME} etc. for simple naming of targets, libraries, etc. it makes it hard to navigate.
The only things I notice are:
-
libpyne.sodoes not have an equivalentRPATH, but it seems to point to a system library so it should be fine without it. - The only
RPATHfortransport_spatial_methods.cpython-311-x86_64-linux-gnu.soseems to point tolibpyne.so. But I also don't see theinstalldirective forpynetarget to check if it's installed incore/libor other paths that you've added there.
Another thing you can do is to try and build it outside of SKBUILD, and with some PYTHONPATH manipulation (point it to the root of package/__init__.py, the built python_add_library etc., maybe check this snippet) and try to run these in the build directory. If that fails, the issue is not on the install side.
You are right. The issue was occurring due to main.f, not from the installation side, and it is now resolved.
However, before closing this issue, I have a question to ask: is there any variable available from the scikit-build side that performs a similar function to the one below?
set(SKBUILD_LIB_DIR ${CMAKE_INSTALL_LIBDIR}/python${Python_VERSION_MAJOR}.${Python_VERSION_MINOR}/site-packages/${install-dir}/${CMAKE_INSTALL_LIBDIR})
I know there is SKBUILD_PLATLIB_DIR, but I think it contains the full path. Also, I don't know if there any variable that contains ${install-dir}.
You cannot install to site-packages directly when making a wheel. You must install to the wheel and then let pip or installer unpack it into site-packages.
Actually, I am not planning to install directly from the CMake. I will be using this for setting up RPATH, from env/bin to install-dir/lib.
Here is my current approach:
set (SKBUILD_LIB_DIR ${CMAKE_INSTALL_LIBDIR}/python${Python_VERSION_MAJOR}.${Python_VERSION_MINOR}/site-packages/pymoab/core/${CMAKE_INSTALL_LIBDIR})
# Set the install RPATH
if(APPLE)
set_target_properties(compareFiles PROPERTIES INSTALL_RPATH "@loader_path/../${SKBUILD_LIB_DIR}")
elseif(UNIX)
set_target_properties(compareFiles PROPERTIES INSTALL_RPATH "$ORIGIN/../${SKBUILD_LIB_DIR}")
endif()
I’ll have to look, I’m worried about making assumptions that might not always be true. FYI FindPython provides the site packages dir as Python_SITEARCH (https://cmake.org/cmake/help/latest/module/FindPython.html).
Unfortunately, this also sets the full path.
Example:
/usr/local/lib/python3.11/dist-packages
I am looking for something like:
lib/python3.11/dist-packages
More precisely:
lib/python3.11/dist-packages/${install-dir}/lib
I think you can get the ROOT dir and compute it with cmake's relative path computation. But I'm pretty sure this is not something you can put in a wheel. When you set up Python, you can configure these values. dist-packages is not always separate from site-packages. And if you were to ever use ABI3, the Python number could change.
In general, I think you can't make links between the data dir and the platlib dir. It's also fairly unfriendly to users to put things in data, since if they don't use virtual environments, they are getting things installed directly to /usr/local or wherever Python consider's it's root. Most packages put everything in platlib, then all paths are guaranteed to be relocatable when the wheel is installed. If you do put things in data or bin, it shouldn't depend on finding platlib unless it can query it from Python at runtime.
I am closing this as I am solving the issue using the following method.
# Scikit-build installs MOAB to ${SKBUILD_PLATLIB_DIR}/pymoab/core
# So, set bin directory to root environment (install prefix/bin)
set(CMAKE_INSTALL_BINDIR ${SKBUILD_SCRIPTS_DIR})
set(SKBUILD_LIB_DIR ${CMAKE_INSTALL_LIBDIR}/python${Python_VERSION_MAJOR}.${Python_VERSION_MINOR}/site-packages/pymoab/core/${CMAKE_INSTALL_LIBDIR})
# Auditwheel and Delocate need this when repairing the wheel
# Since MOAB libs are installed in the pymoab/core subdirectory
# Auditwheel and Delocate need this to find the MOAB libs for pymoab.data
# It's a bit of a hack, but it works
set(SKBUILD_REPAIR_WHEEL_PATCH pymoab/core/${CMAKE_INSTALL_LIBDIR})
if(APPLE)
set(PYMOAB_LIBRARY_RPATH "@loader_path")
set(PYMOAB_PYTHON_MODULE_RPATH "@loader_path/core/${CMAKE_INSTALL_LIBDIR}")
set(PYMOAB_BINARY_RPATH "@loader_path/../${SKBUILD_LIB_DIR};@loader_path/../${SKBUILD_REPAIR_WHEEL_PATCH};@loader_path/../../${SKBUILD_REPAIR_WHEEL_PATCH}")
elseif(UNIX)
set(PYMOAB_LIBRARY_RPATH "$ORIGIN")
set(PYMOAB_PYTHON_MODULE_RPATH "$ORIGIN/core/${CMAKE_INSTALL_LIBDIR}")
set(PYMOAB_BINARY_RPATH "$ORIGIN/../${SKBUILD_LIB_DIR};$ORIGIN/../${SKBUILD_REPAIR_WHEEL_PATCH};$ORIGIN/../../${SKBUILD_REPAIR_WHEEL_PATCH}")
else()
set(PYMOAB_LIBRARY_RPATH OFF)
set(PYMOAB_PYTHON_MODULE_RPATH OFF)
set(PYMOAB_BINARY_RPATH OFF)
endif()