[kdtree] Doesn't create a lib when compiling with MSVC

[SunBlack]

trafficstars

I tried to upgrade the PCL version which our CI is using for Windows from 1.11.1 to 1.12.1, but sadly this fails as somehow the build of pcl_kdtree creates a dll, but no lib. With 1.11.1 I have no issue, but with 1.12.0, 1.12.1 and the current master.

Tried it on two machines: Both with Win10, the on machine is using MSVC 19.16 (2017), the other 19.29 (2019) and 19.32 (2022).

I broke our call to CMake a bit down:

CMake step

Note: The MSVC_MP=1 is to make the build log better understandable. And BUILD_tools=OFF to shorten the build log.

C:\dev\dependencies\pcl_src\build>cmake .. -G "Visual Studio 16 2019" -A x64 -DCMAKE_INSTALL_PREFIX=../../pcl -DPCL_NO_PRECOMPILE=ON -DBUILD_kdtree=ON -DBUILD_tools=OFF -DMSVC_MP=1
-- Selecting Windows SDK version 10.0.19041.0 to target Windows 10.0.19044.
-- The C compiler identification is MSVC 19.29.30146.0
-- The CXX compiler identification is MSVC 19.29.30146.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Enterprise/VC/Tools/MSVC/14.29.30133/bin/Hostx64/x64/cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Enterprise/VC/Tools/MSVC/14.29.30133/bin/Hostx64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Could NOT find PkgConfig (missing: PKG_CONFIG_EXECUTABLE)
-- Found ClangFormat: C:/Program Files/LLVM/bin/clang-format.exe (found suitable version "13.0.0", minimum required is "10")
-- Adding target 'format'
-- Performing Test HAVE_MM_MALLOC
-- Performing Test HAVE_MM_MALLOC - Failed
-- Performing Test HAVE_POSIX_MEMALIGN
-- Performing Test HAVE_POSIX_MEMALIGN - Failed
-- Performing Test HAVE_SSE4_2_EXTENSIONS
-- Performing Test HAVE_SSE4_2_EXTENSIONS - Success
-- Performing Test HAVE_AVX2
-- Performing Test HAVE_AVX2 - Success
-- Found OpenMP_C: -openmp (found version "2.0")
-- Found OpenMP_CXX: -openmp (found version "2.0")
-- Found OpenMP: TRUE (found version "2.0") found components: C CXX
-- Found OpenMP, spec date 200203
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - not found
-- Check if compiler accepts -pthread
-- Check if compiler accepts -pthread - no
-- Found Threads: TRUE
-- Found Eigen: C:/dev/dependencies/eigen/include/eigen3 (Required is at least version "3.1")
-- Eigen found (include: C:/dev/dependencies/eigen/include/eigen3, version: 3.3.9)
-- Found flann version 1.9.1
-- Could NOT find libusb (missing: libusb_LIBRARIES libusb_INCLUDE_DIR)
-- Could NOT find libusb (missing: LIBUSB_LIBRARIES LIBUSB_INCLUDE_DIRS)
-- Could NOT find OpenNI (missing: OPENNI_LIBRARIES OPENNI_INCLUDE_DIRS)
-- OpenNI grabber support: not building because OpenNI not found
-- Could NOT find OpenNI2 (missing: OPENNI2_LIBRARIES OPENNI2_INCLUDE_DIRS)
-- OpenNI2 grabber support: not building because OpenNI2 not found
-- Could NOT find Ensenso (missing: ENSENSO_LIBRARIES ENSENSO_INCLUDE_DIRS)
-- IDS-Imaging Ensenso camera support: not building because Ensenso not found
-- Could NOT find davidSDK (missing: DAVIDSDK_LIBRARY DAVIDSDK_INCLUDE_DIR)
-- David Vision Systems SDK support: not building because davidSDK not found
-- Could NOT find DSSDK (missing: DSSDK_LIBRARIES DSSDK_INCLUDE_DIRS)
-- DepthSense SDK support: not building because DSSDK not found
-- Could NOT find RSSDK (missing: RSSDK_LIBRARIES RSSDK_INCLUDE_DIRS)
-- RealSense SDK support: not building because RSSDK not found
-- Could NOT find RSSDK2
-- RealSense SDK 2.0 (librealsense) support: not building because RSSDK2 not found
-- Found ZLIB: C:/dev/dependencies/osg-3rdparty/lib/zlib.lib (found version "1.2.11")
-- Found PNG: C:/dev/dependencies/osg-3rdparty/lib/libpng.lib (found version "1.6.37")
-- Could NOT find Qhull (missing: QHULL_LIBRARY QHULL_INCLUDE_DIR)
-- Found CUDA Toolkit v11.7
-- The CUDA compiler identification is NVIDIA 11.7.64
-- Detecting CUDA compiler ABI info
-- Detecting CUDA compiler ABI info - done
-- Check for working CUDA compiler: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.7/bin/nvcc.exe - skipped
-- Detecting CUDA compile features
-- Detecting CUDA compile features - done
-- CMAKE_CUDA_ARCHITECTURES: 35;37;50;52;53;60;61;62;70;72;75;80;86;87
CMake Warning at cmake/pcl_find_vtk.cmake:25 (find_package):
  By not providing "FindVTK.cmake" in CMAKE_MODULE_PATH this project has
  asked CMake to find a package configuration file provided by "VTK", but
  CMake did not find one.

  Could not find a package configuration file provided by "VTK" with any of
  the following names:

    VTKConfig.cmake
    vtk-config.cmake

  Add the installation prefix of "VTK" to CMAKE_PREFIX_PATH or set "VTK_DIR"
  to a directory containing one of the above files.  If "VTK" provides a
  separate development package or SDK, be sure it has been installed.
Call Stack (most recent call first):
  CMakeLists.txt:375 (include)


-- Qt version: 5.15.2
-- Could NOT find Pcap (missing: PCAP_LIBRARIES PCAP_INCLUDE_DIRS)
-- Found Boost: C:/dev/dependencies/boost (found suitable version "1.75.0", minimum required is "1.65.0") found components: filesystem date_time iostreams system regex
-- Found OpenGL: opengl32
-- Could NOT find GLEW (missing: GLEW_INCLUDE_DIR GLEW_LIBRARY)
-- DOXYGEN_FOUND
-- HTML_HELP_COMPILER
-- Found CPack generators: NSIS
-- PCL build with following flags:
-- /DWIN32 /D_WINDOWS /W3 /GR /EHsc /fp:precise /wd4800 /wd4521 /wd4251 /wd4275 /wd4305 /wd4355  /arch:AVX2 /bigobj -openmp
-- The following subsystems will be built:
--   common
--   kdtree
--   octree
--   search
--   sample_consensus
--   filters
--   2d
--   geometry
--   io
--   features
--   ml
--   segmentation
--   surface
--   registration
--   keypoints
--   tracking
--   recognition
--   stereo
-- The following subsystems will not be built:
--   visualization: VTK was not found.
--   apps: Disabled by default
--   benchmarks: Disabled by default
--   outofcore: Disabled manually.
--   examples: Code examples are disabled by default.
--   people: Disabled manually.
--   simulation: Disabled by default.
--   global_tests: Disabled by default
--   tools: Disabled manually.
-- Configuring done
-- Generating done
-- Build files have been written to: C:/dev/dependencies/pcl_src/build

Build step

C:\dev\dependencies\pcl_src\build>cmake --build . --config Release
Microsoft (R) Build Engine version 16.11.2+f32259642 for .NET Framework
Copyright (C) Microsoft Corporation. All rights reserved.

  Building Custom Rule C:/dev/dependencies/pcl_src/common/CMakeLists.txt
  point_types.cpp
  pcl_base.cpp
  PCLPointCloud2.cpp
  io.cpp
  common.cpp
  correspondence.cpp
  distances.cpp
  parse.cpp
  poses_from_matches.cpp
  print.cpp
  projection_matrix.cpp
  time_trigger.cpp
  gaussian.cpp
  colors.cpp
  feature_histogram.cpp
  bearing_angle_image.cpp
  range_image.cpp
  range_image_planar.cpp
  kiss_fft.c
C:\dev\dependencies\pcl_src\common\src\fft\kiss_fft.c(119,53): warning C4244: '=': conversion from 'double' to 'float', possible loss of data [C:\dev\dependencies\pcl_src\build\common\pcl_common.vcxproj]
C:\dev\dependencies\pcl_src\common\src\fft\kiss_fft.c(120,53): warning C4244: '=': conversion from 'double' to 'float', possible loss of data [C:\dev\dependencies\pcl_src\build\common\pcl_common.vcxproj]
C:\dev\dependencies\pcl_src\common\src\fft\kiss_fft.c(224,37): warning C4267: '+=': conversion from 'size_t' to 'int', possible loss of data [C:\dev\dependencies\pcl_src\build\common\pcl_common.vcxproj]
  kiss_fftr.c
C:\dev\dependencies\pcl_src\common\src\fft\kiss_fftr.c(116,46): warning C4244: '=': conversion from 'double' to 'float', possible loss of data [C:\dev\dependencies\pcl_src\build\common\pcl_common.vcxproj]
C:\dev\dependencies\pcl_src\common\src\fft\kiss_fftr.c(117,46): warning C4244: '=': conversion from 'double' to 'float', possible loss of data [C:\dev\dependencies\pcl_src\build\common\pcl_common.vcxproj]
C:\dev\dependencies\pcl_src\common\src\fft\kiss_fftr.c(118,52): warning C4244: '=': conversion from 'double' to 'float', possible loss of data [C:\dev\dependencies\pcl_src\build\common\pcl_common.vcxproj]
C:\dev\dependencies\pcl_src\common\src\fft\kiss_fftr.c(119,52): warning C4244: '=': conversion from 'double' to 'float', possible loss of data [C:\dev\dependencies\pcl_src\build\common\pcl_common.vcxproj]
     Creating library C:/dev/dependencies/pcl_src/build/lib/pcl_common.lib and object C:/dev/dependencies/pcl_src/build/lib/pcl_common.exp
  Generating code
  Finished generating code
  pcl_common.vcxproj -> C:\dev\dependencies\pcl_src\build\bin\pcl_common.dll
  Building Custom Rule C:/dev/dependencies/pcl_src/kdtree/CMakeLists.txt
  kdtree_flann.cpp
  Generating code
  Finished generating code
LINK : warning LNK4199: /DELAYLOAD:VCOMP140.dll ignored; no imports found from VCOMP140.dll [C:\dev\dependencies\pcl_src\build\kdtree\pcl_kdtree.vcxproj]
  pcl_kdtree.vcxproj -> C:\dev\dependencies\pcl_src\build\bin\pcl_kdtree.dll
  Building Custom Rule C:/dev/dependencies/pcl_src/octree/CMakeLists.txt
  octree_inst.cpp
C:/dev/dependencies/boost\boost/detail/iterator.hpp(13): message : This header is deprecated. Use <iterator> instead. [C:\dev\dependencies\pcl_src\build\octree\pcl_octree.vcxproj]
     Creating library C:/dev/dependencies/pcl_src/build/lib/pcl_octree.lib and object C:/dev/dependencies/pcl_src/build/lib/pcl_octree.exp
  Generating code
  Finished generating code
LINK : warning LNK4199: /DELAYLOAD:VCOMP140.dll ignored; no imports found from VCOMP140.dll [C:\dev\dependencies\pcl_src\build\octree\pcl_octree.vcxproj]
  pcl_octree.vcxproj -> C:\dev\dependencies\pcl_src\build\bin\pcl_octree.dll
  Building Custom Rule C:/dev/dependencies/pcl_src/sample_consensus/CMakeLists.txt
  sac.cpp
  sac_model_circle.cpp
  sac_model_circle3d.cpp
  sac_model_cylinder.cpp
  sac_model_cone.cpp
  sac_model_line.cpp
  sac_model_parallel_line.cpp
  sac_model_stick.cpp
  sac_model_normal_parallel_plane.cpp
  sac_model_normal_plane.cpp
  sac_model_normal_sphere.cpp
  sac_model_plane.cpp
  sac_model_registration.cpp
  sac_model_sphere.cpp
     Creating library C:/dev/dependencies/pcl_src/build/lib/pcl_sample_consensus.lib and object C:/dev/dependencies/pcl_src/build/lib/pcl_sample_consensus.exp
  Generating code
  Finished generating code
LINK : warning LNK4199: /DELAYLOAD:VCOMP140.dll ignored; no imports found from VCOMP140.dll [C:\dev\dependencies\pcl_src\build\sample_consensus\pcl_sample_consensus.vcxproj]
  pcl_sample_consensus.vcxproj -> C:\dev\dependencies\pcl_src\build\bin\pcl_sample_consensus.dll
  Building Custom Rule C:/dev/dependencies/pcl_src/search/CMakeLists.txt
  search.cpp
  kdtree.cpp
  brute_force.cpp
  organized.cpp
  octree.cpp
LINK : fatal error LNK1181: cannot open input file '..\lib\pcl_kdtree.lib' [C:\dev\dependencies\pcl_src\build\search\pcl_search.vcxproj]
  Building Custom Rule C:/dev/dependencies/pcl_src/io/CMakeLists.txt
  ply_parser.cpp
     Creating library C:/dev/dependencies/pcl_src/build/lib/pcl_io_ply.lib and object C:/dev/dependencies/pcl_src/build/lib/pcl_io_ply.exp
  Generating code
  Finished generating code
LINK : warning LNK4199: /DELAYLOAD:VCOMP140.dll ignored; no imports found from VCOMP140.dll [C:\dev\dependencies\pcl_src\build\io\pcl_io_ply.vcxproj]
  pcl_io_ply.vcxproj -> C:\dev\dependencies\pcl_src\build\bin\pcl_io_ply.dll
  Building Custom Rule C:/dev/dependencies/pcl_src/io/CMakeLists.txt
  debayer.cpp
  pcd_grabber.cpp
C:\dev\dependencies\pcl_src\io\include\pcl/io/low_level_io.h(97,39): warning C4267: 'argument': conversion from 'size_t' to 'unsigned int', possible loss of data [C:\dev\dependencies\pcl_src\build\io\pcl_io.vcxproj]
C:\dev\dependencies\pcl_src\io\include\pcl/io/low_level_io.h(102,40): warning C4267: 'argument': conversion from 'size_t' to 'unsigned int', possible loss of data [C:\dev\dependencies\pcl_src\build\io\pcl_io.vcxproj]
  pcd_io.cpp
  vtk_io.cpp
  ply_io.cpp
  ascii_io.cpp
  compression.cpp
  lzf.cpp
  lzf_image_io.cpp
C:\dev\dependencies\pcl_src\io\include\pcl/io/low_level_io.h(97,39): warning C4267: 'argument': conversion from 'size_t' to 'unsigned int', possible loss of data [C:\dev\dependencies\pcl_src\build\io\pcl_io.vcxproj]
C:\dev\dependencies\pcl_src\io\include\pcl/io/low_level_io.h(102,40): warning C4267: 'argument': conversion from 'size_t' to 'unsigned int', possible loss of data [C:\dev\dependencies\pcl_src\build\io\pcl_io.vcxproj]
  obj_io.cpp
  ifs_io.cpp
  image_grabber.cpp
  hdl_grabber.cpp
  Please define _WIN32_WINNT or _WIN32_WINDOWS appropriately. For example:
  - add -D_WIN32_WINNT=0x0601 to the compiler command line; or
  - add _WIN32_WINNT=0x0601 to your project's Preprocessor Definitions.
  Assuming _WIN32_WINNT=0x0601 (i.e. Windows 7 target).
  vlp_grabber.cpp
  Please define _WIN32_WINNT or _WIN32_WINDOWS appropriately. For example:
  - add -D_WIN32_WINNT=0x0601 to the compiler command line; or
  - add _WIN32_WINNT=0x0601 to your project's Preprocessor Definitions.
  Assuming _WIN32_WINNT=0x0601 (i.e. Windows 7 target).
  robot_eye_grabber.cpp
  Please define _WIN32_WINNT or _WIN32_WINDOWS appropriately. For example:
  - add -D_WIN32_WINNT=0x0601 to the compiler command line; or
  - add _WIN32_WINNT=0x0601 to your project's Preprocessor Definitions.
  Assuming _WIN32_WINNT=0x0601 (i.e. Windows 7 target).
  auto_io.cpp
  io_exception.cpp
  tim_grabber.cpp
  Please define _WIN32_WINNT or _WIN32_WINDOWS appropriately. For example:
  - add -D_WIN32_WINNT=0x0601 to the compiler command line; or
  - add _WIN32_WINNT=0x0601 to your project's Preprocessor Definitions.
  Assuming _WIN32_WINNT=0x0601 (i.e. Windows 7 target).
  libpng_wrapper.cpp
     Creating library C:/dev/dependencies/pcl_src/build/lib/pcl_io.lib and object C:/dev/dependencies/pcl_src/build/lib/pcl_io.exp
  Generating code
  Finished generating code
  pcl_io.vcxproj -> C:\dev\dependencies\pcl_src\build\bin\pcl_io.dll
  Building Custom Rule C:/dev/dependencies/pcl_src/ml/CMakeLists.txt
  point_xy_32i.cpp
  point_xy_32f.cpp
  densecrf.cpp
  pairwise_potential.cpp
  permutohedral.cpp
  svm_wrapper.cpp
  svm.cpp
  kmeans.cpp
     Creating library C:/dev/dependencies/pcl_src/build/lib/pcl_ml.lib and object C:/dev/dependencies/pcl_src/build/lib/pcl_ml.exp
  Generating code
  Finished generating code
LINK : warning LNK4199: /DELAYLOAD:VCOMP140.dll ignored; no imports found from VCOMP140.dll [C:\dev\dependencies\pcl_src\build\ml\pcl_ml.vcxproj]
  pcl_ml.vcxproj -> C:\dev\dependencies\pcl_src\build\bin\pcl_ml.dll
  Building Custom Rule C:/dev/dependencies/pcl_src/stereo/CMakeLists.txt
  stereo_grabber.cpp
  stereo_matching.cpp
  stereo_block_based.cpp
  stereo_adaptive_cost_so.cpp
  disparity_map_converter.cpp
  digital_elevation_map.cpp
     Creating library C:/dev/dependencies/pcl_src/build/lib/pcl_stereo.lib and object C:/dev/dependencies/pcl_src/build/lib/pcl_stereo.exp
  Generating code
  Finished generating code
LINK : warning LNK4199: /DELAYLOAD:VCOMP140.dll ignored; no imports found from VCOMP140.dll [C:\dev\dependencies\pcl_src\build\stereo\pcl_stereo.vcxproj]
  pcl_stereo.vcxproj -> C:\dev\dependencies\pcl_src\build\bin\pcl_stereo.dll

The error message is just:

LINK : fatal error LNK1181: cannot open input file '..\lib\pcl_kdtree.lib' [C:\dev\dependencies\pcl_src\build\search\pcl_search.vcxproj]

Sadly the pcl_kdtree build itself is successfully.

Until now I have no idea, what causes the issue.

[mvieth]

Not sure, maybe you can use git bisect to find the problematic commit between 1.11.1 and 1.12.0?

[SunBlack]

git bisect is indeed a helpful command 👍 I tracked it now down to the MR #4358 which introduced the issue. This MR removes #include <pcl/common/io.h> from kdtree/include/pcl/kdtree/kdtree.h. With this include it compiles on my machine. I tracked this include further down and it seems #include <pcl/PCLPointCloud2.h> is needed somehow, even I don't understand why until now (including just the includes which PCLPointCloud2.h includes isn't enough, so it must sth within the file which is needed).

Note: It is enough to compile common, kdtree, octree and search to reproduce the issue.

[SunBlack]

It seems #4358 is causing other problems in the PCL to come to the surface rather than being the problem itself.

When compiling the commit before #4358 and inspecting the pcl_kdtree.lib via dumpbin.exe -headers pcl_kdtree.lib you will see, that only the class PCLPointCloud2 is exported. This explains the issue as by removing the include nothing have to be exported and therefore MSVC omits the lib.

So my conclusion by seeing this:

• In case PCL_NO_PRECOMPILE is ON, the module kdtree should be marked as interface library as nothing is getting exported
• There are currently exports within pcl_kdtree.lib which should not be there. This should be fixed by fixing #2708.
• When PCL_NO_PRECOMPILE is OFF the build currently succeed as workaround (not checked vcpkg, but this could be the reason why 1.12.0 works there).

[mvieth]

So maybe something like this?

set(EXT_DEPS flann)
if(PCL_NO_PRECOMPILE)
    add_library(${LIB_NAME} INTERFACE)
    target_link_libraries("${LIB_NAME}" INTERFACE pcl_common FLANN::FLANN)
    PCL_MAKE_PKGCONFIG(${LIB_NAME} COMPONENT ${SUBSYS_NAME} DESC ${SUBSYS_DESC} PCL_DEPS ${SUBSYS_DEPS} EXT_DEPS ${EXT_DEPS} HEADER_ONLY)
else()
    PCL_ADD_LIBRARY(${LIB_NAME} COMPONENT ${SUBSYS_NAME} SOURCES ${srcs} ${incs} ${impl_incs})
    target_link_libraries("${LIB_NAME}" pcl_common FLANN::FLANN)
    PCL_MAKE_PKGCONFIG(${LIB_NAME} COMPONENT ${SUBSYS_NAME} DESC ${SUBSYS_DESC} PCL_DEPS ${SUBSYS_DEPS} EXT_DEPS ${EXT_DEPS})
endif()

Can you test if this works in your case and then perhaps create a pull request?

[SunBlack]

In principle yes, even if we should extend PCL_ADD_LIBRARY so that he can trade it, where I would not give it the keyword INTERFACE, but rather PRECOMPILED_SOURCES, then you would kill 2 birds with one stone:

• If there are no more sources, he could switch to interfaces alone and the logic for that would be central
• If PCL_NO_PRECOMPILE is specified and the source file contains a definition only for this, one would not need to show it to the compiler at all. This might have a very difficult measurable effect on the compile speed, but it doesn't hurt.