mujoco icon indicating copy to clipboard operation
mujoco copied to clipboard

LTO is forcefully enabled instead of by default in the build system

Open davidhozic opened this issue 2 months ago • 5 comments

Intro

Hi!

I am a student (in a few days researcher) at University of Ljubljana, I use MuJoCo for my research on RL.

My setup

MuJoCo 3.3.7. Other info is irrelevant.

What's happening? What did you expect?

A few CMake files contain forceful set of the CMake variable CMAKE_INTERPROCEDURAL_OPTIMIZATION (LTO).

  • https://github.com/google-deepmind/mujoco/blob/36bed9a9da8cbd42a8dddd8ab3d586a51abedd7d/cmake/MujocoOptions.cmake#L107-L109
  • https://github.com/google-deepmind/mujoco/blob/36bed9a9da8cbd42a8dddd8ab3d586a51abedd7d/sample/cmake/SampleOptions.cmake#L107-L109
  • https://github.com/google-deepmind/mujoco/blob/36bed9a9da8cbd42a8dddd8ab3d586a51abedd7d/simulate/cmake/SimulateOptions.cmake#L107-L109

This causes problems whenever the compiled (static) libraries may be used with a different linker, which is now the case for the Rust language on the default Linux target (they switched to rust-lld).

As a result of the forceful enable, the compiled library files contain additional information that is only compatible with the same linker. See the following issue: https://github.com/rust-lang/rust/issues/146952.

Based on the commit message that added the forceful enable of LTO, the plan was to enable it by default, not permanently.

Several CI files also contain commands where this is explicitly disabled (https://github.com/google-deepmind/mujoco/blob/36bed9a9da8cbd42a8dddd8ab3d586a51abedd7d/.github/workflows/build.yml#L178), but as the result of the given lines inside CMake option files, nothing changes.

Steps for reproduction

  1. Compile with CMAKE_INTERPROCEDURAL_OPTIMIZATION disabled from the console
  2. CMAKE_INTERPROCEDURAL_OPTIMIZATION (LTO) is still enabled

Minimal model for reproduction

Irrelevant

Code required for reproduction

Irrelevant

Confirmations

davidhozic avatar Oct 17 '25 15:10 davidhozic

Would it be suitable to just remove those lines? I tried changing them to a CACHE variable but then everything just fails with no explanation (I guess CMake special options don't support CACHE).

davidhozic avatar Oct 17 '25 16:10 davidhozic

Can we change the conditional to if(NOT DEFINED CMAKE_INTERPROCEDURAL_OPTIMIZATION...)?

saran-t avatar Oct 21 '25 17:10 saran-t

Can we change the conditional to if(NOT DEFINED CMAKE_INTERPROCEDURAL_OPTIMIZATION...)?

That won't work because this variable is defined by default.

I think the best way to only enable it by default is by making another CMake option with a default value and then setting the CMAKE_INTERPROCEDURAL_OPTIMIZATION to that option.

Edit:

I tried using if(NOT DEFINED CMAKE_INTERPROCEDURAL_OPTIMIZATION and it is confirmed that the variable is defined by default so adding an option, like described above, should work best.

It would just need to be documented somewhere that CMAKE_INTERPROCEDURAL_OPTIMIZATION is not to be used.

davidhozic avatar Oct 21 '25 18:10 davidhozic

Maybe this can work?

option(MUJOCO_ENABLE_LTO ON)
if(MUJOCO_ENABLE_LTO AND CMAKE_BUILD_TYPE AND NOT CMAKE_BUILD_TYPE STREQUAL "Debug")
  set(CMAKE_INTERPROCEDURAL_OPTIMIZATION ON)
endif()

Yoshi-Sama-lab avatar Nov 01 '25 16:11 Yoshi-Sama-lab

Maybe this can work?

option(MUJOCO_ENABLE_LTO ON)
if(MUJOCO_ENABLE_LTO AND CMAKE_BUILD_TYPE AND NOT CMAKE_BUILD_TYPE STREQUAL "Debug")
  set(CMAKE_INTERPROCEDURAL_OPTIMIZATION ON)
endif()

That's not alright, as it ignores LTO value on Debug builds. The pull request #2913 solves this properly.

davidhozic avatar Nov 01 '25 17:11 davidhozic