Copy adolc and dependencies to install folder on linux

[aymanhab]

Fixes issue #<issue_number>

Brief summary of changes

Testing I've completed

Looking for feedback on...

CHANGELOG.md (choose one)

• no need to update because...
• updated.

---

This change is 

[aymanhab]

@adamkewley I started working on this a while back but got stuck trying to find a replacement for install_name_tool on unix that can be generic/available out of the box. If you can help here since you're more familiar with linux tools that would be great. Thank you

[aymanhab]

@adamkewley We're trying to finalize 4.2, if you can take a look and fix or suggest a solution so that the linux build is usable that would be much appreciated. Thank you

[aymanhab]

@mitkof6 Can you help with this PR so that linux builds are functional? Thanks

[mitkof6]

Hi @aymanhab. Sorry, I just spotted this. I would be glad to help, but I am not sure that I understood the issue. Do you want to pack adolc within the OpenSim's install directory? Why not have it as an external dependency?

[aymanhab]

Hi @mitkof6, the issue is not with building but with making a distribution on linux/ubuntu. We copy all the necessary files to the install directory already but the loader-path for the .so files is not modified, which causes a distribution or other forms of delivery to fail to find the dependencies. On osx, there's install_name_tool that we use as described above, I'm trying to find a similar tool readily available on linux to do the same (change rpath for so's to the install folder). Let me know if you have any questions or suggestions. Typically applications are self contained so no external dependencies, and there's no way to guarantee version/compiler compatibility. Thank you

[mitkof6]

I see. I am not sure how things work on Mac, but on Linux, the philosophy is a bit different. We typically set the LD_LIBRARY_PATH to point to the new folder where the libraries are copied before running the application. This will make sure (prefer) to load the libraries located in this folder without overriding the relative paths. The drawback is that we will have to set LD_LIBRARY_PATH every time you call the application. Alternatively, there is a nice discussion that suggests the use of patchelf or chrpath as alternatiaves.

[mitkof6]

Also, CMake has the CMAKE_BUILD_WITH_INSTALL_RPATH. This is set in combination with CMAKE_INSTALL_RPATH. Do you think this might be useful?

[aymanhab]

Thanks @mitkof6 this could be helpful though may need a more recent version of CMake than we currently require. Will keep you posted.

[mitkof6]

I am looking into this. Linux does not have the same restrictions as MAC. I will try to translate the commands.

[aymanhab]

@AlbertoCasasOrtiz This PR should take care of adolc related issues on linux (both copying and rpath setting) Please review and comment as needed. In particular if you can check for both python and java/Matlab/GUI on linux that would be great. Thank you

[aymanhab]

@AlbertoCasasOrtiz Can you please confirm that Ld_library_path is not needed to run the command line executables from this artifact on linux? Thank you

[AlbertoCasasOrtiz]

@aymanhab I just tested this in a clean Ubuntu 18.04 environment and it is still showing the following error:

./opensim-cmd: error while loading shared libraries: libadolc.so.2: cannot open shared object file: No such file or directory

After installing it, it is still showing it.

[adamkewley]

@AlbertoCasasOrtiz, just to unnecessarily geg-in on this, because been a while since I worked on this kind of thing (my daily driver is currently Windows, because I'm doing GUI stuff), but iirc the issue on Linux is that (and I may be wrong here - it's been a while) adolc is built separately (e.g. while building the dependencies) using a separate build system (e.g. Makefiles) and the way that CMake tends to work is that:

• For dev, it builds stuff as-is and hard-codes the library paths as something like an absolute path in each library (e.g. libosimcomon.so will literally have a dependency on /home/adam/Desktop/adolc/adolc.so)
• For installation, it re-builds the binaries with a relative path (e.g. libosimcommon.so will have a dependency on @rpath/../lib/adolc.so). It then copies the binaries to the installation location

For these sub-builds there's a risk that CMake just copies the development-friendly binaries over without changing the relative library paths. It should handle this, because this PR (for example) is using CMake's install command, but (again, my memory is bad) it might require some encouragement.

The easiest way to debug the issue is to install something like lddtree and run it on your binaries (e.g. lddtree opensim-cmd). Everything in that list should, for an installation build, use relative paths. This is because downstream users can install binaries wherever they please (if you want a portable installation, that is).

Actually fixing it requires a bit of elbow grease. I don't know about adolc, but I had to mess around a little bit with CMake to create a portable installation of my downstream app (osc):

• e.g. telling CMake to use an appropriate rpath https://github.com/ComputationalBiomechanicsLab/opensim-creator/blob/main/CMakeLists.txt#L744
• after setting that var, telling CMake to copy the file: https://github.com/ComputationalBiomechanicsLab/opensim-creator/blob/main/CMakeLists.txt#L937
• (I think that CMake then detects that it's installing relevant binaries and sets up the rpath accordingly)

[aymanhab]

@AlbertoCasasOrtiz going back to finish this, the current status is that the libraries in question are copied but we end up with broken links for libgfortran and libquadmath and also the rpath is not updated. The latter is due to the difference in usage/syntax between otool on mac and patchelf on linux. I'll sort the latter out but would like to know the layout that works on osx for (gfortran and quadmath) and I don't have access to a mac at the moment, can you help? Thanks

[aymanhab]

@AlbertoCasasOrtiz since you just done this for osx (copy and fix rpath for tropter libraries on osx), can you pickup this PR on linux to do the same? Thank you

[AlbertoCasasOrtiz]

@aymanhab Sure, I'll work on this.

[AlbertoCasasOrtiz]

@aymanhab I think I just solved this issue in this branch https://github.com/opensim-org/opensim-core/tree/fix_linux_library_problems

I will integrate the changes in this PR and better document my changes, and the reason of the problem as soon as I arrive today. I will test on Ubuntu 18, 20, 22 and Debian 10 and 11.

[AlbertoCasasOrtiz]

@aymanhab With the latest commits, I have just tested this on a clean Ubuntu 22.04 environment and it is working. No warnings/errors related to libraries appear when starting the GUI.

In order to test this, I have a fork of opensim-gui that obtains the appropriate branch for opensim-core while building. Here is the generated artifacts with this configuration: https://github.com/AlbertoCasasOrtiz/opensim-gui/actions/runs/3970774891

[AlbertoCasasOrtiz]

@aymanhab I have tested this on Ubuntu 18.04, 20.04 and 22.04. It is working now, but there is an issue with Ubuntu 18.04. Basically, looks like the issue we had on Google colab with the conda package. Builds created with Ubuntu 20.04 or above do not work on Ubuntu 18.04. The build created on Ubuntu 18.04 works perfectly.

https://github.com/opensim-org/opensim-core/issues/3369

[aymanhab]

@AlbertoCasasOrtiz Sorry to tell you this but the artifact I downloaded still shows the same problems on a fresh linux on windows. I'd leave this open. Thanks for all your efforts on fixing this rather nasty issue.

[aymanhab]

@AlbertoCasasOrtiz I think this is now fixed, but will wait for a fresh ci build before declaring victory ✌️

[aymanhab]

Downloading and testng artifact on ubuntu 20.04 I can run opensim-cmd and opensense executables from arbitrary folders. Since these require loading the whole stack of libraries at runtime will declare victory and merge. Big thanks @AlbertoCasasOrtiz 🥇