[question] Replacing cmake-conan with CMakeToolchain+CMakeDeps

[Nekto89]

As cmake-conan is considered legacy and doesn't work well with cross-compilation I'm thinking how to migrate. I need to be able to call conan multiple times with different lists of refs. I thought about this approach:

1. generate profile before calling CMake in any way
2. conan install -g CMakeToolchain -pr:b default -pr:h generated_profile .
3. pass generated toolchain to CMake generation and also path to host profile
4. instead of using cmake-conan run "conan install -g CMakeDeps ...." with generated host profile, dynamic list of refs and other options inside additional profiles
5. call find_package for each package

Will this theoretically work?

[Nekto89]

I also see that profile information is available in conan.lock in the same folder as generated toolchain. So I can probably parse it instead of passing paths to profiles through additional CMake options.

[prince-chrismc]

Is there a reason do two conan installs? You can pass multiple generators at ones?

Lockfiles got a major rework, the 2.0 probably dont have the settings from what I've learnt so far 🤔 They are just a list of versions rather then the graph (like in 1.x)

[Nekto89]

Is there a reason do two conan installs? You can pass multiple generators at ones?

I have complex structure of multiple separate projects that are added through add_subdirectory. Each one has its own cmake structure and consists out of tens/hundreds libraries that can be individually enabled/disabled. Calling conan multiple times currently looks like the easiest way. I've reduced number of calls but having one call would be very hard.

Lockfiles got a major rework, the 2.0 probably dont have the settings from what I've learnt so far 🤔 They are just a list of versions rather then the graph (like in 1.x)

So it's better not to depend on this for future... Maybe making custom generator that just dumps final profile_host/profile_build might be a good idea?

[hwhsu1231]

Maybe making custom generator that just dumps final profile_host/profile_build might be a good idea?

@Nekto89

I do think it is better to use Host and Build profiles with cmake-conan since this method is recommended in Conan 2.0. Therefore, I already proposed a feature request issue and a PR to fulfill a new command called conan_cmake_profile(). I'm still waiting this PR to be merged.

• Issue: https://github.com/conan-io/cmake-conan/issues/450
• PR: https://github.com/conan-io/cmake-conan/pull/451

There is a simple usage of conan_cmake_profile() with conan_cmake_install() in https://github.com/conan-io/cmake-conan/issues/450#issuecomment-1303722392.

[hwhsu1231]

I thought about this approach:

1. generate profile before calling CMake in any way
2. conan install -g CMakeToolchain -pr:b default -pr:h generated_profile.
3. pass generated toolchain to CMake generation and also path to host profile
4. instead of using cmake-conan run "conan install -g CMakeDeps ...." with generated host profile, dynamic list of refs and other options inside additional profiles
5. call find_package for each package

Will this theoretically work?

@Nekto89

In my opinion, it is better to make CMake as the starting point instead of Conan in terms of the integration of Conan with CMake. However, there was a CAE problem when using CMakeToolchain generator. You can check the details in https://github.com/conan-io/cmake-conan/issues/410#issuecomment-1095611994. This problem made Conan Team decide to deprecate cmake-conan in Conan 2.0 at one time. You can check the details in https://github.com/conan-io/cmake-conan/issues/447#issuecomment-1271688707. Therefore, in order to solve this CAE problem and make Conan Team keep maintaining cmake-conan, I provided an alternative way to use cmake-conan.

• Issue: https://github.com/conan-io/cmake-conan/issues/449
• Project: cmake-conan-setup

[prince-chrismc]

Maybe making custom generator that just dumps final profile_host/profile_build might be a good idea?

That's actually how the conan profile show will work in 2.0 😄 it takes the same CLI as install or create and will spit the final one so you don't have to

[hwhsu1231]

@prince-chrismc

So what are the differences of conan profile show commands between 1.0 and 2.0? According to Conan 1.0 documentation, the purpose of using conan profile show command is to show the values defined for a profile.

[prince-chrismc]

Please do not use bold (or caps) in messages. The convention on the internet chat is that means you are screaming. It comes across you with wrong tone.

I'd suggest playing with 2.0 so you understand better.

It still takes a profile and spits out the settings - but it also has the "default" build profile (in 2.0 default its based on the conf) and you can pass multiple profiles and settings and anything else you want

[hwhsu1231]

Please do not use bold (or caps) in messages. The convention on the internet chat is that means you are screaming. It comes across you with wrong tone.

Thanks for reminding me. I'll keep it in mind. I thought adding bold or caps can emphasize the important keywords.

[hwhsu1231]

@prince-chrismc

So back to the original problem, I really think Conan should provide a usage method in which CMake is the starting point instead of Conan. I saw you give a "thumbs down" emoji to the comment where I mention cmake-conan-setup. I want to know what's your opnions about cmake-conan? Do you think it should be deprecated?

[prince-chrismc]

Do you think it should be deprecated?

I really like this question! Because it's absolutely wrong in all the right ways 🤣 . This repo is not deprecated and as it's explained in the README.

This supporting project is not the priority - the client needs to work - there's 10s of thousands of Conan users that are waiting for the 2.0 launch who need those improvements.

As always we absolutely want to hear about how you are using Conan, we take in ALL the feedback from GitHub, Slack, talking directly to customers who use it a scale that's impossible to comprehend- we take all of it and make the best call possible to benefit everyone.

---

This project is useless if Conan does not work. (My personal opinion having relied on it professionally). As a result that needs to be finished and delivered. Once that is done, we will hopefully have more bandwidth to look into solving the challenges here.

This is on the agenda for 2.x and we will take all the feedback from the last 4 years and make the right decision.

Hopefully that clears how why cmake-conan is NOT deprecated.

The work you are doing on your own is very interesting and it's great you are sharing it. it's been noted 😉

Keep up the great work.

[calebkiage]

I have explored FetchContent and vcpkg, but I found the build times on FetchContent be slow, vcpkg was better with its cache of dependencies outside the project, that helped with the download phase, but I couldn't specify dependency versions easily. Conan seemed best suited for the workflow I desired. I just started using conan and found the conan install... then cmake... to be a little hard to get used to as a consumer of packages. To make my workflow easier, I found a way that worked with CMake + conan using chained toolchain files. I have a toolchain always available that runs conan install for me and generates the conan toolchain file. I then include the conan toolchain at the end of my toolchain. That has largely worked but I think there's potential to make it easier. I have a very small POC in this repo that anyone is free to look at. I'd love to see good docs on what the ideal way to use conan with CMake for package consumers would be.

[memsharded]

Thanks very much @calebkiage

Indeed, the idea of doing the installation in a file outside of the CMakeLists.txt looks much better to me. Your repo will indeed be useful, it is a very interesting approach, I think it might be similar to some of the other proposals I have seen above, so certainly this approach could be gaining traction. Looking forward to getting 2.0 out so we can put some time on this too. Cheers!

[duncanspumpkin]

I have explored FetchContent and vcpkg, but I found the build times on FetchContent be slow, vcpkg was better with its cache of dependencies outside the project, that helped with the download phase, but I couldn't specify dependency versions easily. Conan seemed best suited for the workflow I desired. I just started using conan and found the conan install... then cmake... to be a little hard to get used to as a consumer of packages. To make my workflow easier, I found a way that worked with CMake + conan using chained toolchain files. I have a toolchain always available that runs conan install for me and generates the conan toolchain file. I then include the conan toolchain at the end of my toolchain. That has largely worked but I think there's potential to make it easier. I have a very small POC in this repo that anyone is free to look at. I'd love to see good docs on what the ideal way to use conan with CMake for package consumers would be.

I tried out your idea and I've a few ideas of how to improve it.

It would be better to take the install step outside of the toolchain file. This will ensure the toolchain file can be used correctly as CMake will use the toolchain file for processing things outside of the main cmake binary area (such as when doing try_compiles). As long as the install step happens prior to the first call of project within the CMakeLists.txt then it all works fine. The project function call is what will trigger the toolchain to come into scope.

My idea is the main CMakeLists.txt would end up something like this:

cmake_minimum_required(VERSION 3.24)
include(ConanCMakeIntegration) # This step will install all the packages and create the conan_toolchain.cmake

project(blah LANGUAGES CXX) # conan_toolchain.cmake is now in use

find_package(zlib)

ConanCMakeIntegration.cmake would be roughly similar to your already existing code. You just need to then add the conan_toolchain.cmake to your presets and it all works with minimal complexity.

[calebkiage]

@duncanspumpkin, the toolchain already handles that. It checks for the existance of a conanfile before running conan install.

[duncanspumpkin]

One new issue i came across is how to handle build tools and their environment. Say for example you have the following:

1. wxWidgets wxrc build tool built with expat, zlib shared;
2. cmake add_custom_command that converts an .xrc file into a .cpp file at build time

This will fail as the wxrc tool will not be able to see the expat or zlib dynamic libraries. So the issue is that conanbuildenv is not being run prior to a add_custom_command. I guess this is just a regular issue with conan rather than related to CMakeToolchain though.

[crsib]

The very same applies to Qt. However I don't think that this problem is related to cmake-conan. For Qt we had to tweak the recipe so it creates "portable" packages :-)

[duncanspumpkin]

My current solution is to augment all of the add_custom_commands with:

add_custom_command(...
    COMMAND ${CONAN_BUILD_ENV}
    COMMAND ${wxWidgets_wxrc_EXECUTABLE}
     ...)

where CONAN_BUILD_ENV is just pointing to the conanbuild.sh/.bat file generated by the install.

[memsharded]

A proposal for 2.0 based on dependency providers and providing transparent (no need to modify CMakeLists.txt) is in https://github.com/conan-io/cmake-conan/tree/develop2, testing and feedback welcome.

[Nekto89]

A proposal for 2.0 based on dependency providers and providing transparent (no need to modify CMakeLists.txt) is in https://github.com/conan-io/cmake-conan/tree/develop2, testing and feedback welcome.

I've briefly looked through suggested workflow and I have some questions.

1. Do I understand correctly that this will just allow to automatically call conan once for first find_package?
2. Can CMakeDeps be used without CMakeToolchain? I understand that there might be potential differences in profile, but overall is it safe to do so?
3. If answers for 1 and 2 are yes... Can I just continue using cmake-conan v1 approach and just adjust all conan calls for v2? This will allow me to call conan multiple times and change parameters on the fly in different places.

[memsharded]

Hi @Nekto89

1. Yes. This will call conan install ... the first time a find_package() is found in CMakeLists.txt. It will only call conan install one, the result is cached and subsequent find_package() will reuse it
2. Yes and no, it depends on the context. When you are building some project as a developer, then you can run only cmake ... and with the conan_provider.cmake it will call Conan, deriving the configuration from the current CMake execution. So it will not need to define a CMakeToolchain, because the driver is CMake, and the alignment will be achieved by defining the Conan settings from the current CMake settings (the opposite of what CMakeToolchain does). But when a package is being created with Conan (conan create . or conan build .), then Conan is the driver, the Conan input profile needs to be mapped to CMake configuration, and in this case the CMakeToolchain is mandatory. In this cases, the conan_provider.cmake will not be used at all.
3. Not sure if I understand. The new approach is totally different to the 1.X one, for a start it doesn't need to modify the CMakeLists.txt at all, so there are no "conan call" to adjust here.

[Nekto89]

Not sure if I understand. The new approach is totally different to the 1.X one, for a start it doesn't need to modify the CMakeLists.txt at all, so there are no "conan call" to adjust here.

I already have completely working solution on slightly patched cmake-conan v1+conan 1.X.X. I don't have any problems with changing my cmake files - conan is already deeply integrated in the whole CMake generation step on multiple layers. I just want to switch to conan 2.X.X because CCI will soon start breaking compatibility with cmake_find_package_multi which was working fine for me. So the easiest way to do the switch will be to rewrite part in cmake-conan v1 that calls conan and maybe slightly adjust work with profiles.

[Nekto89]

Not sure if I understand. The new approach is totally different to the 1.X one, for a start it doesn't need to modify the CMakeLists.txt at all, so there are no "conan call" to adjust here.

I already have completely working solution on slightly patched cmake-conan v1+conan 1.X.X. I don't have any problems with changing my cmake files - conan is already deeply integrated in the whole CMake generation step on multiple layers. I just want to switch to conan 2.X.X because CCI will soon start breaking compatibility with cmake_find_package_multi which was working fine for me. So the easiest way to do the switch will be to rewrite part in cmake-conan v1 that calls conan and maybe slightly adjust work with profiles.

I basically need:

1. automatically try to convert current cmake options to conan profile as much as possible
2. allow adding/removing/editing any values there
3. be able to call "conan install" any number of times with any profiles/references/generators and output result to specified folder

[memsharded]

automatically try to convert current cmake options to conan profile as much as possible

This is what the current conan_provider.cmake is implementing, automatic conversion of cmake config to Conan profile. Users are contributing automatic mapping for Android, iOS, etc. If there is something that is not correctly mapping, the feedback is welcome, the goal is that this mapping is as complete as possible

allow adding/removing/editing any values there

This hasn't been defined yet. The idea is that when CMake is driving the build, all configuration comes from CMake, so it shouldn't be necessary to customize much. So we are focusing first on the automatic mapping, and then soon we will start designing how the customization will work, but it will take some time

be able to call "conan install" any number of times with any profiles/references/generators and output result to specified folder

This is not planned, as cmake can only handle multi-configuration for Release/Debug/others, in some generators, like Visual Studio, that will be the variability that will be managed. So Conan will be able to call conan install for dependencies for Debug/Release/others in one single cmake invocation. But calling conan install for other variations in a single cmake run is not expected, because cmake cannot handle it, and that will require different cmake calls.