pinocchio icon indicating copy to clipboard operation
pinocchio copied to clipboard

Unexpected -O2 flag when building on Debug with Conda

Open abussy-aldebaran opened this issue 1 year ago • 6 comments

Hi,

Bug description

When building pinocchio in Debug, on Ubuntu, using the conda environment installed through

conda env update -n base -f pinocchio/.github/workflows/conda/environment_macos_linux.yml

the -O2 flag is set (among other flags).

Expected behavior

The -O2 flag is not set.

Reproduction steps

It can be seen on pinocchio CI with a failed Debug build job. For example: https://github.com/stack-of-tasks/pinocchio/actions/runs/11182992289/job/31090536335#step:7:1979

The probable origin of this bug can be seen by executing env with the conda environment activated:

$ env
...
CXXFLAGS=-fvisibility-inlines-hidden -fmessage-length=0 -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/my_user/miniforge3/envs/pinocchio/include
DEBUG_CXXFLAGS=-fvisibility-inlines-hidden -fmessage-length=0 -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-all -fno-plt -Og -g -Wall -Wextra -fvar-tracking-assignments -ffunction-sections -pipe -isystem /home/my_user/miniforge3/envs/pinocchio/include
...

Additional context

While debugging my application using pinocchio, I saw some variables optimized out. After investigating, I found out that my debug build was using the -O2 flag. I'm pretty sure it comes from the environment variables set by conda. Searching on the web, I found: https://github.com/conda/conda/issues/10195 https://github.com/conda/conda-build/issues/3481 https://docs.conda.io/projects/conda-build/en/latest/user-guide/environment-variables.html But my understanding of conda is too basic to know what to do of it.

So I tried to find if pinocchio's CI was handling this and how, and I found out it isn't.

Screenshots

From pinocchio CI, https://github.com/stack-of-tasks/pinocchio/actions/runs/11182992289/job/31090536335#step:7:1979 image

System

  • OS: Ubuntu with Conda
  • Pinocchio version: devel branch

abussy-aldebaran avatar Oct 09 '24 11:10 abussy-aldebaran

These are the default compilation flags of conda. Have you checked that? This seems not related to Pinocchio.

jcarpent avatar Oct 09 '24 11:10 jcarpent

I agree that the problem comes from conda, not Pinocchio. But it seems to me that Pinocchio's CI is impacted. I'm going to keep investigating to find a clean solution for my project. Could you please let me know if you find one for Pinocchio's CI ?

abussy-aldebaran avatar Oct 09 '24 12:10 abussy-aldebaran

Yes, you’re correct. I will let @jorisv answer the question.

jcarpent avatar Oct 09 '24 12:10 jcarpent

@abussy-aldebaran One quick solution could be to clear from the env variable CXXFLAGS the O2 option. On my system, I have:

-ftree-vectorize -fPIC -fstack-protector-strong -O2 -pipe -stdlib=libc++ -fvisibility-inlines-hidden -fmessage-length=0 -isystem /opt/homebrew/Caskroom/miniconda/base/envs/general-latest-eigen/include

after calling:

echo $CXXFLAGS

jcarpent avatar Oct 09 '24 13:10 jcarpent

I found: https://github.com/conda-forge/ctng-compiler-activation-feedstock/issues/80 https://github.com/conda-forge/conda-forge.github.io/issues/2002 https://conda-forge.org/docs/user/faq/#faq-compiler-required-options

Apparently, the meta-package cxx-compiler is doing too much hand-holding:

The conda-forge infrastructure provides activation scripts which are run when you conda activate an environment that contains the compiler toolchain. Those scripts set many environment variables that are typically used by GNU autotools and make in the standard (i.e. builtin) build rules. For example, you would see the variable CC set to the long compiler name x86_64-conda-linux-gnu-cc. The activation scripts also set a CMAKE_ARGS variable with many arguments the conda-forge community finds helpful for configuring cmake build flows.

If I understand correctly, a solution would be to install gxx and conda-gcc-specs. But it wouldn't work for macOs, right ?

abussy-aldebaran avatar Oct 09 '24 14:10 abussy-aldebaran

Hello @abussy-aldebaran,

We are aware of this issue and I find it really annoying. Like you said, one solution could be to avoid installing cxx-compiler. But this package is really convenient (install the right default compiler per OS).

I'm working to add a pixi support to all our project. Pixi is a package manager that use conda package but also add a lot of nice feature. On of them is to allow an activation script to run when activating the pixi environment. In this PR, I use this activation script to unset CXXFLAGS (https://github.com/stack-of-tasks/eigenpy/pull/444/files#diff-963bc921e6166b5aea3f26481499a05fd92db338cdbd106fa49a87845babca71) and avoid user to have this issue.

jorisv avatar Oct 14 '24 08:10 jorisv

Hello @abussy-aldebaran,

The Pixi PR is merged #2459

If you use Pixi to manage your dependencies you will not have this issue anymore.

jorisv avatar Oct 28 '24 15:10 jorisv

Thanks @jorisv !

Pixi only uses project-wide environments, right ? My use case is to install Pinocchio from source (https://stack-of-tasks.github.io/pinocchio/download.html). I use conda to install the dependencies. So I guess that I can't use Pixi.

abussy-aldebaran avatar Nov 15 '24 13:11 abussy-aldebaran

Yes, I you want to build pinocchio and install it in another environment, then pixi is not tailored to your need. Maybe when pixi build will be available, we will have a solution to this use case.

You can still use pixi to generate the environment.yml with all needed dependencies:

pixi project export conda-environment

Then, before building, you can source development/scripts/pixi/activation.sh to reset all conda CXX flags.

jorisv avatar Nov 15 '24 14:11 jorisv