Document how to handle the compilation from source if multiple MATLAB are found in the system

[nunoguedelha]

Issue description

After reinstalling the superbuild in a new folder and trying to use iDynTree bindings in MATLAB, I get the error:

>> iDynTree.StringVector()

Invalid MEX-file '/Users/nunoguedelha/dev/PR-testing-2/robotology-superbuild/build/install/mex/iDynTreeMEX.mexmaci64':
dlopen(/Users/nunoguedelha/dev/PR-testing-2/robotology-superbuild/build/install/mex/iDynTreeMEX.mexmaci64, 6): Symbol not found: __ZN8iDynTree10SolidShape18setLink_H_geometryERKNS_9TransformE
  Referenced from: /Users/nunoguedelha/dev/PR-testing-2/robotology-superbuild/build/install/mex/../lib/libidyntree-solid-shapes.dylib
  Expected in: /Users/nunoguedelha/dev/PR-testing/robotology-superbuild/build/install/lib/libidyntree-model.dylib
 in /Users/nunoguedelha/dev/PR-testing-2/robotology-superbuild/build/install/mex/../lib/libidyntree-solid-shapes.dylib

Error in iDynTree.StringVector (line 57)
        tmp = iDynTreeMEX(36, varargin{:});

Steps for reproducing the issue

1. I had an old superbuild installed in /Users/nunoguedelha/dev/PR-testing/robotology-superbuild/ and working fine with MATLAB.
2. I downloaded a new superbuild from scratch to /Users/nunoguedelha/dev/PR-testing-2/robotology-superbuild/.
3. Did the cmake configuration, updated the ~/.bash_profile, sourced it again, built everything.
4. Run MATLAB 2020b from /Users/nunoguedelha/Documents/MATLAB
5. In MATLAB, tried to run iDynTree.StringVector(), getting the error.

[nunoguedelha]

Prior analysis

A prior look to the error message and a quick check on the DY libraries caught my attention.

I the error message we see:

Symbol not found: __ZN8iDynTree10SolidShape18setLink_H_geometryERKNS_9TransformE Referenced from: /Users/nunoguedelha/dev/PR-testing-2/robotology-superbuild/build/install/mex/../lib/libidyntree-solid-shapes.dylib Expected in: /Users/nunoguedelha/dev/PR-testing/robotology-superbuild/build/install/lib/libidyntree-model.dylib

Meaning that the executable iDynTreeMEX.mexmaci64 is trying to load libidyntree-solid-shapes.dylib, which references iDynTree::SolidShape::setLink_H_geometry(iDynTree::Transform const&) by looking at /Users/nunoguedelha/dev/PR-testing/robotology-superbuild/build/install/lib/libidyntree-model.dylib, which is a dependency encoded in /Users/nunoguedelha/dev/PR-testing-2/robotology-superbuild/build/install/mex/../lib/libidyntree-solid-shapes.dylib.

$ cd /Users/nunoguedelha/dev/PR-testing-2/robotology-superbuild/build
$ nm -P install/lib/libidyntree-solid-shapes.dylib | grep setLink_H_geometry | grep __ZN8iDynTree10SolidShape18setLink_H_geometryERKNS_9TransformE
__ZN8iDynTree10SolidShape18setLink_H_geometryERKNS_9TransformE U 0 0

$ nm --demangle install/lib/libidyntree-solid-shapes.dylib | grep setLink_H_geometry
                 U iDynTree::SolidShape::setLink_H_geometry(iDynTree::Transform const&)

The executable should be looking in the new superbuild folder PR-testing-2, not the old one PR-testing.

A few checks

I've used the otool Mach-O disassembler for checking the dependencies:

$ otool -L install/lib/libidyntree-solid-shapes.dylib
install/lib/libidyntree-solid-shapes.dylib:
	@rpath/libidyntree-solid-shapes.dylib (compatibility version 0.0.0, current version 0.0.0)
	@rpath/libidyntree-model.dylib (compatibility version 0.0.0, current version 0.0.0)
	@rpath/libidyntree-core.dylib (compatibility version 0.0.0, current version 0.0.0)
	/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 904.4.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1292.60.1)

How is @rpath resolved in @rpath/libidyntree-model.dylib? It is encoded as load commands in libidyntree-solid-shapes.dylib:

$ otool -l install/lib/libidyntree-solid-shapes.dylib
...
Load command 11
          cmd LC_LOAD_DYLIB
      cmdsize 56
         name @rpath/libidyntree-model.dylib (offset 24)
   time stamp 2 Thu Jan  1 01:00:02 1970
      current version 0.0.0
compatibility version 0.0.0
...
Load command 18
          cmd LC_RPATH
      cmdsize 32
         path @loader_path/ (offset 12)
Load command 19
          cmd LC_RPATH
      cmdsize 32
         path @loader_path/../lib (offset 12)

and @loader_path should be the parent folder of the running plugin or executable. So I don't see how we end up looking into Users/nunoguedelha/dev/PR-testing/robotology-superbuild/build/install.

[nunoguedelha]

Wrong MATLAB version selected

After erasing iDynTreeMEX.mexmaci64 from Users/nunoguedelha/dev/PR-testing/robotology-superbuild/build/install/mex/, the error disappeared for some time, but MATLAB crashed at some point, after what the error occurred again.

I then realised that iDynTreeMEX.mexmaci64 from Users/nunoguedelha/dev/PR-testing-2/robotology-superbuild/build/install/mex/` contained the wrong MATLAB version in the Applications path, as it was pointing to 2019b instead of 2020b:

$ otool -l iDynTreeMEX.mexmaci64
...
Load command 31
          cmd LC_RPATH
      cmdsize 32
         path @loader_path/ (offset 12)
Load command 32
          cmd LC_RPATH
      cmdsize 32
         path @loader_path/../lib (offset 12)
Load command 33
          cmd LC_RPATH
      cmdsize 56
         path /Applications/MATLAB_R2019b.app/bin/maci64 (offset 12)
Load command 34
          cmd LC_RPATH
      cmdsize 64
         path /Applications/MATLAB_R2019b.app/extern/bin/maci64 (offset 12)

Actually I've noticed some issues in the iDynTree cmake configuration

Wrong MATLAB version in the iDynTree cmake configuration

MATLAB_2020b release is missing from https://github.com/robotology/idyntree/blob/fced0b0d6635d8a1eeb87ae98e4e8e853928657a/cmake/FindMatlab.cmake#L244-L267:

set(MATLAB_VERSIONS_MAPPING
  "R2020a=9.8"
  "R2019b=9.7"
  "R2019a=9.6"
   ...

In addition, although the environment variable Matlab_ROOT_DIR was set properly in the ~/.bash_profile:

export Matlab_ROOT_DIR=/Applications/MATLAB_R2020b.app
export MATLAB_DIR=$Matlab_ROOT_DIR/bin

it was not propagated to the cmake config of iDynTree where Matlab_ROOT_DIR was still pointing to MATLAB 2019b.

Using MATLAB_ADDITIONAL_VERSIONS compile variable in the cmake command didn't work

I tried to build iDynTree with the option MATLAB_ADDITIONAL_VERSIONS="R2020b=9.9" by configuring cmake in build/robotology/iDynTree as follows:

cd $ROBOTOLOGY_SUPERBUILD_SOURCE_DIR
ccmake ./ -DMATLAB_ADDITIONAL_VERSIONS="R2020b=9.9" -G Xcode

This didn't update the Matlab_ROOT_DIR in the cmake config (ccmake ./ -G Xcode in iDynTree) as we would expect.

Manually changing Matlab_ROOT_DIR

After changing it manually in the cmake config through ccmake, all the variables containing the MATLAB path were updated accordingly, as well as the loading commands in iDynTreeMEX.mexmaci64:

$ otool -l install/mex/iDynTreeMEX.mexmaci64
...
Load command 31
          cmd LC_RPATH
      cmdsize 32
         path @loader_path/ (offset 12)
Load command 32
          cmd LC_RPATH
      cmdsize 32
         path @loader_path/../lib (offset 12)
Load command 33
          cmd LC_RPATH
      cmdsize 56
         path /Applications/MATLAB_R2020b.app/bin/maci64 (offset 12)
Load command 34
          cmd LC_RPATH
      cmdsize 64
         path /Applications/MATLAB_R2020b.app/extern/bin/maci64 (offset 12)

After that, relaunched MATLAB and everything has been working properly since then.

@traversaro what do you think of this? I will opten an issue in iDynTree for my last observations on MATLAB_ADDITIONAL_VERSIONS and Matlab_ROOT_DIR.

CC @lrapetti @kouroshD

[traversaro]

Sorry @nunoguedelha, I am missing a bit the context: Why iDynTree was finding MATLAB 2019b? Did you had two different MATLAB installation on your system? Unfortunately CMake's FindMATLAB caches several variables related to MATLAB, so if you migrate to a new version of MATLAB you need to create a new build directory.

[nunoguedelha]

Yes @traversaro , indeed I had still the old MATLAB version installed. But if I recall well, I had created the new robotology-superbuild after migrating to MATLAB_2020b. I'm not sure though.

So if it wasn't for the cmake problem of caching variables related to MATLAB, cmake's should have set automatically Matlab_ROOT_DIR? Why don't we use the Matlab_ROOT_DIR environment variable for this?

[traversaro]

Yes @traversaro , indeed I had still the old MATLAB version installed. But if I recall well, I had created the new robotology-superbuild after migrating to MATLAB_2020b. I'm not sure though.

The point is that if both MATLAB 2020b and 2019b are installed, I am not sure which one CMake will use, and this is probably creating some of the problems. In that case, you need to specify in some way to CMake which MATLAB to use.

So if it wasn't for the cmake problem of caching variables related to MATLAB, cmake's should have set automatically Matlab_ROOT_DIR? Why don't we use the Matlab_ROOT_DIR environment variable for this?

Note that the FindMATLAB is something available in CMake upstream, and so we need to align to what they do there. More in particular, note that the Matlab_ROOT_DIR (as documented in https://cmake.org/cmake/help/v3.19/module/FindMatlab.html) is not an environment variable, but just a normal CMake variable, so if you need to pass it to all the CMake subprojects of the superbuild, you can pass to the superbuild it via the YCM_EP_ADDITIONAL_CMAKE_ARGS option (see https://github.com/robotology/robotology-superbuild#how-do-i-pass-cmake-options-to-the-projects-built-by-the-robotology-superbuild-) i.e.:

cmake -DYCM_EP_ADDITIONAL_CMAKE_ARGS:STRING="-DMatlab_ROOT_DIR:<TYPE>=<value>"

We could document this better, a bit similar to how in https://github.com/robotology/yarp/issues/2509#issuecomment-785845587 we mentioned that we should document better how to specify the Python3 to use if multiple are installed in the environment.

[nunoguedelha]

Thanks a lot @traversaro . I'll take a look at waht is being done in https://github.com/robotology/yarp/issues/2509 next week.

[traversaro]

Renamed the issue to clarify the definition of done.