cmake-conan

cmake-conan copied to clipboard

I have faced with issue that cmake-conan detect toolchain as gcc and when it tries to install library it is compiled to native Linux target, but actually I need it to be compiled to target ESP32

Here is my toolchain:

-- CMAKE_C_COMPILER_LAUNCHER <PATH>/espressif/espressif_tools/tools/xtensa-esp32-elf/esp-2020r2-8.2.0/xtensa-esp32-elf/bin/xtensa-esp32-elf-gcc
-- CMAKE_CXX_COMPILER_LAUNCHER <PATH>/espressif/espressif_tools/tools/xtensa-esp32-elf/esp-2020r2-8.2.0/xtensa-esp32-elf/bin/xtensa-esp32-elf-g++
-- CMAKE_C_COMPILER_AR <PATH>/espressif/espressif_tools/tools/xtensa-esp32-elf/esp-2020r2-8.2.0/xtensa-esp32-elf/bin/xtensa-esp32-elf-gcc-ar
-- CMAKE_CXX_COMPILER_AR <PATH>/espressif/espressif_tools/tools/xtensa-esp32-elf/esp-2020r2-8.2.0/xtensa-esp32-elf/bin/xtensa-esp32-elf-gcc-ar

Also cmake-conan do not support Generic OS and it is an issue for microcontrollers because usually OS may not be available ...

I think it is possible to add here: https://github.com/conan-io/cmake-conan/blob/develop/conan.cmake#L109

Generic platform

I also created small patch for it https://github.com/conan-io/cmake-conan/pull/285

Hi @redradist,

If I understood correctly, CMake is not using the compiler specified in the toolchain but the default one? I have tried using this toolchain (there are some example cmake toolchains in the installation directory of the ESP32 development environment)

set(CMAKE_SYSTEM_NAME Generic)

set(CMAKE_C_COMPILER <PATH>/tools/xtensa-esp32-elf/esp-2020r3-8.4.0/xtensa-esp32-elf/bin/xtensa-esp32-elf-gcc)
set(CMAKE_CXX_COMPILER <PATH>//tools/xtensa-esp32-elf/esp-2020r3-8.4.0/xtensa-esp32-elf/bin/xtensa-esp32-elf-g++)
set(CMAKE_ASM_COMPILER <PATH>/tools/xtensa-esp32-elf/esp-2020r3-8.4.0/xtensa-esp32-elf/bin/xtensa-esp32-elf-gcc)

set(CMAKE_C_FLAGS "-mlongcalls -Wno-frame-address" CACHE STRING "C Compiler Base Flags")
set(CMAKE_CXX_FLAGS "-mlongcalls -Wno-frame-address" CACHE STRING "C++ Compiler Base Flags")

And it seems to work (ignoring the check for CMAKE_SYSTEM_NAME in conan.cmake if the variable is set to Generic). Maybe your issue is related to using the CMAKE_<LANG>_COMPILER_LAUNCHER var instead of CMAKE_<LANG>_COMPILER?

Hi @redradist,

If I understood correctly, CMake is not using the compiler specified in the toolchain but the default one? I have tried using this toolchain (there are some example cmake toolchains in the installation directory of the ESP32 development environment)

set(CMAKE_SYSTEM_NAME Generic)

set(CMAKE_C_COMPILER <PATH>/tools/xtensa-esp32-elf/esp-2020r3-8.4.0/xtensa-esp32-elf/bin/xtensa-esp32-elf-gcc)
set(CMAKE_CXX_COMPILER <PATH>//tools/xtensa-esp32-elf/esp-2020r3-8.4.0/xtensa-esp32-elf/bin/xtensa-esp32-elf-g++)
set(CMAKE_ASM_COMPILER <PATH>/tools/xtensa-esp32-elf/esp-2020r3-8.4.0/xtensa-esp32-elf/bin/xtensa-esp32-elf-gcc)

set(CMAKE_C_FLAGS "-mlongcalls -Wno-frame-address" CACHE STRING "C Compiler Base Flags")
set(CMAKE_CXX_FLAGS "-mlongcalls -Wno-frame-address" CACHE STRING "C++ Compiler Base Flags")

And it seems to work (ignoring the check for CMAKE_SYSTEM_NAME in conan.cmake if the variable is set to Generic). Maybe your issue is related to using the CMAKE_<LANG>_COMPILER_LAUNCHER var instead of CMAKE_<LANG>_COMPILER?

@czoido No, compiler by cmake detected proper that issue is that conan.cmake use the following profile:

Configuration:
[settings]
arch=x86_64
arch_build=x86_64
build_type=Debug
compiler=gcc
compiler.libcxx=libstdc++11
compiler.version=8
os=Linux
os_build=Linux
[options]
[build_requires]
[env]

As you can see in this profile wrong arch and arch_build and also there is missed proper toolchain linker, conan.cmake detected the envoriment as for Ubuntu, but it should also read the following variables CMAKE_C_LINKER_PREFERENCE and CMAKE_CXX_LINKER_PREFERENCE and set those values in generated profile

Hi @redradist, I think that if you want that kind of behaviour you will have to add those fields and set the profile to use, conan.cmake does not provide the ability to automatically override those settings in the default profile. So maybe you can try to create your custom profile for crosscompilation (let's call it xcompile_profile) and call conan_cmake_run with the PROFILE argument:

conan_cmake_run(PROFILE xcompile_profile)

If you want to set any environment variable you can do it in the [env] section of your profile. Please, have a look at: https://docs.conan.io/en/latest/systems_cross_building/cross_building.html

Hi @redradist, I think that if you want that kind of behaviour you will have to add those fields and set the profile to use, conan.cmake does not provide the ability to automatically override those settings in the default profile. So maybe you can try to create your custom profile for crosscompilation (let's call it xcompile_profile) and call conan_cmake_run with the PROFILE argument:

conan_cmake_run(PROFILE xcompile_profile)

If you want to set any environment variable you can do it in the [env] section of your profile. Please, have a look at: https://docs.conan.io/en/latest/systems_cross_building/cross_building.html

@czoido But can you explain why conan.cmake does not detect CMAKE_C_LINKER_PREFERENCE and CMAKE_CXX_LINKER_PREFERENCE ? It is pretty easy to implement in conan.cmake ... if those variables are empty then you can use default one in system ...

conan.cmake can set automaticaly these variables:

[env]
CONAN_CMAKE_FIND_ROOT_PATH=$toolchain
CHOST=$target_host
AR=$target_host-ar
AS=$target_host-as
RANLIB=$target_host-ranlib
CC=$target_host-$cc_compiler
CXX=$target_host-$cxx_compiler
STRIP=$target_host-strip
RC=$target_host-windres

I thought the whole reason of existing of conan.cmake is to help user not create profile by hand ...

@czoido But can you explain why conan.cmake does not detect CMAKE_C_LINKER_PREFERENCE and CMAKE_CXX_LINKER_PREFERENCE ? It is pretty easy to implement in conan.cmake ... if those variables are empty then you can use default one in system ...

I think that I don't completely understand what you are expecting conan.cmake to do, how should use those variables and detect them? Maybe that's out of the scope of this script and responds to a more specific behaviour. Can you provide more details on the operations you are doing? How are you invoking cmake? Are you setting the toolchain with -DCMAKE_TOOLCHAIN_FILE?

By the way, you can always add environment info in the conan_cmake_run call, so you don't have to create a profile to do it.

include(conan.cmake)
conan_cmake_run(...
                ENV env_var=value
                ENV Pkg:env_var2=value2)

@czoido But can you explain why conan.cmake does not detect CMAKE_C_LINKER_PREFERENCE and CMAKE_CXX_LINKER_PREFERENCE ? It is pretty easy to implement in conan.cmake ... if those variables are empty then you can use default one in system ...

I think that I don't completely understand what you are expecting conan.cmake to do, how should use those variables and detect them? Maybe that's out of the scope of this script and responds to a more specific behaviour. Can you provide more details on the operations you are doing? How are you invoking cmake? Are you setting the toolchain with -DCMAKE_TOOLCHAIN_FILE?

By the way, you can always add environment info in the conan_cmake_run call, so you don't have to create a profile to do it.

include(conan.cmake)
conan_cmake_run(...
                ENV env_var=value
                ENV Pkg:env_var2=value2)

@czoido I run cmake using Clion like this:

<PATH>/clion-2020.2/bin/cmake/linux/bin/cmake -DCMAKE_BUILD_TYPE=Debug -G Ninja <PROJECT_PATH>
<PATH>/clion-2020.2/bin/cmake/linux/bin/cmake --build <PROJECT_PATH>/cmake-build-debug-ninja --target <PROJECT_NAME>.elf

Here is my CMakeLists.txt:

set(SUPPORTED_TARGETS esp32)
include($ENV{IDF_PATH}/tools/cmake/project.cmake)
project(<PROJECT_NAME> C CXX)

include(${CMAKE_CURRENT_BINARY_DIR}/config/sdkconfig.cmake)

# Download automatically, you can also just copy the conan.cmake file
if(NOT EXISTS "${CMAKE_SOURCE_DIR}/conan.cmake")
    message(STATUS "Downloading conan.cmake from https://github.com/conan-io/cmake-conan")
    file(DOWNLOAD "https://github.com/conan-io/cmake-conan/raw/v0.15/conan.cmake"
            "${CMAKE_SOURCE_DIR}/conan.cmake"
            TLS_VERIFY ON)
endif()

include(${CMAKE_BINARY_DIR}/conan.cmake)

conan_cmake_run(REQUIRES fmt/6.1.2
                BASIC_SETUP
                BUILD missing)

message(STATUS "CONAN_LIBS is ${CONAN_LIBS}")
target_link_libraries(${PROJECT_NAME}.elf ${CONAN_LIBS})

I expect that conan.cmake read all need toolchain information from CMAKE_* (CMAKE_C_COMPILER, CMAKE_C_LINKER, CMAKE_CXX_COMPILER, CMAKE_CXX_LINKER) variables and create proper profile even for cross-compilation use-case

Hi @redradist,

I think you are missing a call to a toolchain that sets CMAKE_C_COMPILER, CMAKE_CXX_COMPILER, CMAKE_ASM_COMPILER and so on. I think that's the way it should be done (https://cmake.org/cmake/help/latest/manual/cmake-toolchains.7.html)

<PATH>/clion-2020.2/bin/cmake/linux/bin/cmake -DCMAKE_BUILD_TYPE=Debug -G Ninja <PROJECT_PATH> -DCMAKE_TOOLCHAIN_FILE=esp_toolchain.cmake

CMake will then have the correct information and after that conan.cmake will set the correct compiler in the profile. For the architecture you will have to set that manually because there's no way that conan.cmake can infer the platform you want to build for unless you provide that information in some way (like it's done in the toolchain file).

Hi @redradist,

I think you are missing a call to a toolchain that sets CMAKE_C_COMPILER, CMAKE_CXX_COMPILER, CMAKE_ASM_COMPILER and so on. I think that's the way it should be done (https://cmake.org/cmake/help/latest/manual/cmake-toolchains.7.html)

<PATH>/clion-2020.2/bin/cmake/linux/bin/cmake -DCMAKE_BUILD_TYPE=Debug -G Ninja <PROJECT_PATH> -DCMAKE_TOOLCHAIN_FILE=esp_toolchain.cmake

CMake will then have the correct information and after that conan.cmake will set the correct compiler in the profile. For the architecture you will have to set that manually because there's no way that conan.cmake can infer the platform you want to build for unless you provide that information in some way (like it's done in the toolchain file).

@czoido No, I did not missed setting toolchain

There is some trick, compilers sets in include($ENV{IDF_PATH}/tools/cmake/project.cmake) this line

Here is results of:

message(STATUS "CMAKE_C_COMPILER is ${CMAKE_C_COMPILER}")
message(STATUS "CMAKE_CXX_COMPILER is ${CMAKE_CXX_COMPILER}")

provides the following results:

-- CMAKE_C_COMPILER is <PATH>/espressif/espressif_tools/tools/xtensa-esp32-elf/esp-2020r2-8.2.0/xtensa-esp32-elf/bin/xtensa-esp32-elf-gcc
-- CMAKE_CXX_COMPILER is <PATH>/espressif/espressif_tools/tools/xtensa-esp32-elf/esp-2020r2-8.2.0/xtensa-esp32-elf/bin/xtensa-esp32-elf-g++

But profile detected by cmake-conan:

[settings]
arch=x86_64
arch_build=x86_64
build_type=Debug
compiler=gcc
compiler.libcxx=libstdc++11
compiler.version=8
os=Linux
os_build=Linux
[options]
[build_requires]
[env]

Yes, that seems fine. So, then, the only thing failing is the detection of the architecture? As you are crosscompiling the only way that the correct architecture will be set is passing it in some way as an argument or a profile...

Yes, that seems fine. So, then, the only thing failing is the detection of the architecture? As you are crosscompiling the only way that the correct architecture will be set is passing it in some way as an argument or a profile...

@czoido Not only detection of architechure, but also seems like used gcc from the system and not one from toolchain

Hi @redradist, The architecture is not going to be passed correctly to cmake conan because there's no way to detect that, maybe if CMAKE_SYSTEM_PROCESSOR was set in the toolchain a detection could be implemented. For the compiler, I see here: https://github.com/conan-io/cmake-conan/issues/284#issuecomment-712787271 in the profile detected that the compiler version and name seems correct (gcc, 8)? Did you check it running <PATH>/espressif/espressif_tools/tools/xtensa-esp32-elf/esp-2020r2-8.2.0/xtensa-esp32-elf/bin/xtensa-esp32-elf-gcc --version?

Hi @redradist, The architecture is not going to be passed correctly to cmake conan because there's no way to detect that, maybe if CMAKE_SYSTEM_PROCESSOR was set in the toolchain a detection could be implemented. For the compiler, I see here: #284 (comment) in the profile detected that the compiler version and name seems correct (gcc, 8)? Did you check it running <PATH>/espressif/espressif_tools/tools/xtensa-esp32-elf/esp-2020r2-8.2.0/xtensa-esp32-elf/bin/xtensa-esp32-elf-gcc --version?

@czoido I have checked version of compiler:

$ ./xtensa-esp32-elf-gcc --version
xtensa-esp32-elf-gcc (crosstool-NG esp-2020r2) 8.2.0
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

$ ./xtensa-esp32-elf-g++ --version
xtensa-esp32-elf-g++ (crosstool-NG esp-2020r2) 8.2.0
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

@czoido I have created pull-request for working on initial support of detection of architecture

https://github.com/conan-io/cmake-conan/pull/293

I expect that conan.cmake read all need toolchain information from CMAKE_* (CMAKE_C_COMPILER, CMAKE_C_LINKER, CMAKE_CXX_COMPILER, CMAKE_CXX_LINKER) variables and create proper profile even for cross-compilation use-case

I was also expecting this, but no, conan.cmake are not reading all needed toolchain informaiton ..... For example see my android solution -> https://github.com/Bjoe/cmake-conan/blob/crosscompile-boost/CMakeLists.txt#L68-L97 It takes all the 'toolchain settings" from the parameter file

cmake -GNinja -DCMAKE_BUILD_TYPE=Release -DANDROID_ABI=arm64-v8a -DANDROID_CPP_FEATURES="rtti exceptions" -DANDROID_PLATFORM=android-28 -DCMAKE_TOOLCHAIN_FILE=/home/joerg/SDKs/Android/ndk/21.3.6528147/build/cmake/android.toolchain.cmake -S cmake-conan -B out/build/cmake-conan-protobuf-test

Maybe for it's sufficient enough todo the same approach as in the issue #307 Infer Conan profile from CMake toolchain file for cross-building?

So PR #191 Auto detect android platform settings and you #293 Initial support for architecure detection are trying so somehow solve the same problem :-)