Failed to build rtabmap with VTK-9.1.0 and PCL-1.12.1

[jiapei100]

/usr/bin/ld: cannot find -lvtkGUISupportQt
/usr/bin/ld: cannot find -lg2o::solver_cholmod
/usr/bin/ld: cannot find -lg2o::solver_csparse
/usr/bin/ld: cannot find -lg2o::csparse_extension

[cdb0y511]

Hi, for vtk you need to enable qt-group support in cmake-gui, and you also need to check g2o cmake information too.

[matlabbe]

This commit should fix the g2o errors.

I'll check for PCL/VTK (longer to test)...

[matlabbe]

Make sure VTK 9.1.0 is built with Qt (not that I tested on Ubutnu bionic, which comes with Qt5.9.5):

• vtk: cmake -DVTK_GROUP_ENABLE_Qt=ON -DVTK_LEGACY_SILENT=ON ..

Things to be careful: if you build dependencies from source, make sure to build all them with or without -march=native flag (edit: also found that SSE and AVX flags would have to be the same too) to avoid Eigen related crashes on runtime (when they are installed from binaries, they are without -march=native).

With -march=native

(Note that PCL built from source is enabling it by default)

• pcl 1.12.1: cmake -DPCL_ENABLE_MARCHNATIVE=ON -DPCL_ENABLE_SSE=ON -DBUILD_tools=OFF ..
• g2o (January 16, 2022): cmake -DBUILD_WITH_MARCH_NATIVE=ON -DG2O_BUILD_APPS=OFF -DG2O_BUILD_EXAMPLES=OFF ..
• gtsam (4.1.1) : ~~cmake -DGTSAM_BUILD_WITH_MARCH_NATIVE=ON -DGTSAM_USE_SYSTEM_EIGEN=ON -DGTSAM_BUILD_EXAMPLES_ALWAYS=OFF -DGTSAM_BUILD_TESTS=OFF ..~~ (Note that I couldn't make GTSAM not crashing on optimization with this mode, maybe related to other sse2/avx flags set by pcl and g2o)

You would have those flags when doing cmake of rtabmap:

cd rtabmap/build
cmake ..
...
--   PCL_COMPILE_OPTIONS = -msse4.2;-mfpmath=sse;-march=native;-mavx2
...

From the VERBOSE=1 build of each dependency: pcl: -msse4.2 -mfpmath=sse -march=native -mavx2 -fopenmp -O2 -g -DNDEBUG -fPIC g2o: -march=native -fPIC -msse2 -msse3 -msse4.1 -msse4.2 gtsam: -O3 -DNDEBUG -fPIC -march=native

Without -march=native, AVX and SSE flags

The most compatible version is by disabling any optimizations:

• pcl 1.12.1: cmake -DPCL_ENABLE_MARCHNATIVE=OFF -DPCL_ENABLE_SSE=OFF -DBUILD_tools=OFF -DPCL_ENABLE_AVX=OFF ..
• g2o (January 16, 2022): cmake -DBUILD_WITH_MARCH_NATIVE=OFF -DG2O_BUILD_APPS=OFF -DG2O_BUILD_EXAMPLES=OFF -DDISABLE_SSE2=ON -DDISABLE_SSE3=ON -DDISABLE_SSE4_1=ON -DDISABLE_SSE4_2=ON -DDISABLE_SSE4_A=ON -DDO_SSE_AUTODETECT=OFF ..
• gtsam (4.1.1): cmake -DGTSAM_BUILD_WITH_MARCH_NATIVE=OFF -DGTSAM_USE_SYSTEM_EIGEN=ON -DGTSAM_BUILD_EXAMPLES_ALWAYS=OFF -DGTSAM_BUILD_TESTS=OFF ..
• ceres (2.0.0): cmake -DBUILD_EXAMPLES=OFF -DBUILT_TESTING=OFF ..

Related issue: https://github.com/introlab/rtabmap/issues/637