DESTDIR not honored for pure cmake packages

[simonschmeisser]

System Info

• Operating System: Ubuntu 16.04
• Python Version: python --version
• Version of catkin_tools:

catkin_tools 0.4.2 (C) 2014-2016 Open Source Robotics Foundation
catkin_tools is released under the Apache License, Version 2.0 (http://www.apache.org/licenses/LICENSE-2.0)

---
Using Python 2.7.11+ (default, Apr 17 2016, 14:00:29) [GCC 5.3.1 20160413]

• ROS Distro: kinetic

Build / Run Issue

I have a workspace containing two pure cmake projects (octomap and fcl) as well as several other catkin packages depending on those. fcl depends on octomap and moveit_core depends on fcl. When I try to use DESTDIR for packaging purposes, it will not set the neccessary include path for fcl to find octomap.

Actual Behavior

Step1: fcl does not find the octomap headers: gist: https://gist.github.com/simonschmeisser/cdde4dd59fb9d48cbf88dcbab780fd3d

Step2: add CPATH to fix includes VERBOSE=1 LANG=c CPATH=/home/sis/ros_upstream2/tmp/opt/mikado-ros/include/ DESTDIR=$(pwd)/tmp catkin build -j4 -p1 -v https://gist.github.com/simonschmeisser/956d9b167c644b46db6edde1ebfb160e

STEP3: add LIBRARY_PATH helps fcl but not collada_urdf VERBOSE=1 LANG=c CPATH=/home/sis/ros_upstream2/tmp/opt/mikado-ros/include/ LIBRARY_PATH=/home/sis/ros_upstream2/tmp/opt/mikado-ros/lib/ DESTDIR=$(pwd)/tmp catkin build -p1 -v https://gist.github.com/simonschmeisser/db78397f63825b2854bd4f1e3f20fad5

[mikepurvis]

The fundamental issue is that CMake packages typically generate installspaces which contain absolute paths and aren't relocatable (unlike catkin packages), and later packages in the same build need to find and use those packages in their build path prior to them being moved to their final location.

The only way to really handle this would be to somehow generate the install metadata twice— once for use during the build (without DESTDIR), and once for the final installation (with DESTDIR).

Alternatively, it could be handled as I do in Clearpath's bundle builds, which is to avoid using DESTDIR, run the whole build to completion, and then as part of the packaging step, execute the following script over the workspace:

#!/bin/bash

ORIGINAL_PATH=$1
NEW_PATH=$2

for f in $(find "$ORIGINAL_PATH" -name *.cmake -o -name *.pc -o -name *.sh -o -name _setup_util.py); do
  sed -i "s|$ORIGINAL_PATH|$NEW_PATH|g" "$f"
done

Maybe catkin_tools could bake in logic such as this, but it's pretty obviously a huge hack. Either way, DESTDIR is broken once plain CMake packages are in the mix (the same applies to catkin_make_isolated, of course).

[simonschmeisser]

I was able to workaround by setting a symlink from NEW_PATH to ORIGINAL_PATH , I guess bind-mount would also work (even while having the package installed)

[leokoppel]

@mikepurvis

The fundamental issue is that CMake packages typically generate installspaces which contain absolute paths and aren't relocatable (unlike catkin packages), and later packages in the same build need to find and use those packages in their build path prior to them being moved to their final location.

Since CMake 3, that functionality is provided by CMAKE_STAGING_PREFIX. In a workspace with only pure CMake packages(e.g. octomap and fcl) I can give catkin config -DCMAKE_STAGING_PREFIX=/a -DCMAKE_INSTALL_PREFIX=/b and everything builds successfully: the files are copied into /a, but contain hardcoded paths for /b.

I think CMake handles this under the nose of catkin_tools, which doesn't know about CMAKE_STAGING_PREFIX. The problem is when I add catkin packages: catkin build attempts to write the setup.sh scripts to CMAKE_INSTALL_PREFIX. If that path isn't writable, the build fails for that package.

Perhaps the solution is simply to respect CMAKE_STAGING_PREFIX?

[mikepurvis]

Interesting, thanks for the info on that! AFAICT from a quick read, for CMAKE_STAGING_PREFIX to work correctly, it will need to be explicitly supported by every CMake find module— otherwise you still have the fundamental issue that the xxx_INCLUDE_DIRS and similar variables will be populated with absolute paths.

[leokoppel]

I don't think it needs to be explicitly supported as long as standard find_* commands are used:

The CMAKE_STAGING_PREFIX location is also used as a search prefix by the find_* commands.

I may be misinterpreting its purpose but I get the idea it is meant to be transparent to packages. The Cmake source may give a better idea of what it does: https://gitlab.kitware.com/cmake/cmake/commit/7521da2852a2fce16919b2640bcc0fa2f0c55b7d

Of course it is possible that packages do something which impedes this feature. For example, Catkin seems to unwittingly override the prefix selection for its setup.* files, which I hope can be fixed like this: https://github.com/leokoppel/catkin/commit/3ece34b70b8b352e85e3cf8188516d45b6ebf6a7

I think this actually a catkin, not catkin_tools issue, though it is relevant to the discussion above.