[question] Use user toolchain file from inside project

[miguelgcasado]

What is your question?

Hi,

I'd like to include a specific toolchain file stored in my project for the conan build. I've read that tools.cmake.cmaketoolchain:user_toolchain should be used but it's not clear to me if it's possible to load a specific toolchain.cmake file and how should be done. Do you have any guidance for this?

Thanks!

Have you read the CONTRIBUTING guide?

• [ ] I've read the CONTRIBUTING guide

[memsharded]

Hi @miguelgcasado

Thanks for your question.

There are more details about the user_toolchain here: https://docs.conan.io/2/reference/tools/cmake/cmaketoolchain.html#using-a-custom-toolchain-file

As a summary:

• You can define it in tool_requires or in profile files, what is your use case?
• You need to define it as a list:
  • define it with a = ["mytoolchain.cmake"] list value
  • or with some append/prepend operator such as tools.cmake.cmaketoolchain:user_toolchain+=my_user_toolchain.cmake (note the +=
• If defined it in profiles, you can use the profile_dir variable to define relative locations, and your mytoolchain.cmake file can be together with the profile file, and can be distributed via conan config install/install-pkg See the example in https://docs.conan.io/2/examples/tools/cmake/cmake_toolchain/inject_cmake_variables.html

  include(default)
  [conf]
  tools.cmake.cmaketoolchain:user_toolchain+={{profile_dir}}/myvars.cmake
  

Please let me know if this helps

[miguelgcasado]

Thanks for the quick response. I don't want to use it in the profiles for facilitating my workflow so I guess it should be in tool_requires. Could you add a small code snippet for prepending this toolchain file to the user_toolchain?

Thanks

[memsharded]

Yes, sure, the one in https://docs.conan.io/2/reference/tools/cmake/cmaketoolchain.html#using-a-custom-toolchain-file:

import os
from conan import ConanFile
class MyToolchainPackage(ConanFile):
    ...
    def package_info(self):
        f = os.path.join(self.package_folder, "mytoolchain.cmake")
        self.conf_info.define("tools.cmake.cmaketoolchain:user_toolchain", [f])

Put that in your "tool-require" recipe, then use it with [tool_requires] in profiles or with self.tool_requires("mytool/version") from other recipes

[miguelgcasado]

If I understand correctly, I'd need to create a separate conan package containing the toolchain file for this, right? Would it be a possibility to do it all in the same conan recipe?

[memsharded]

If I understand correctly, I'd need to create a separate conan package containing the toolchain file for this, right? Would it be a possibility to do it all in the same conan recipe?

The idea of putting it in a different recipe, is because you are going to reuse that recipe as tool_requires for many other packages, including dependencies. If you only want to use a file as a toolchain in just one single package, just put that file together with your project files and CMakeLists.txt. Then you might want just to include() it in the first line of your CMakeLists.txt?

I might be missing what problem you are trying to solve, in principle the toolchains are common files to use in different builds, so not sure what is your use case. For example, using that toolchain in just that one package, but not in the dependencies of that package will likely generate binary incompatibility issues.

[miguelgcasado]

I want to keep parity of the toolchain when compiling the same project by two different methods: plan cmake and conan + cmake, that's why I need to do it this way. And yes, I want to have it applied in the package and also in the dependencies.

[miguelgcasado]

Adding it via --conf=tools.cmake.cmaketoolchain:user_toolchain+=my_user_toolchain.cmake works, but via conanfile.py def generate(self): tc = CMakeToolchain(self) toolchain_path = os.path.abspath(os.path.join(self.source_folder, "Toolchain.cmake")) self.conf_info.append("tools.cmake.cmaketoolchain:user_toolchain", toolchain_path) tc.generate() doesn't seem to work

[memsharded]

I want to keep parity of the toolchain when compiling the same project by two different methods: plan cmake and conan + cmake, that's why I need to do it this way. And yes, I want to have it applied in the package and also in the dependencies.

The best way to apply it project-wide would be:

• Add the conf adding the user toolchain in the profile file, in the [conf] section
• Add it in a tool_requires package, then you can inject that package to all packages adding it to the [tool_requires] section in your profile file.

Yes, both approaches require to use profile files (managed with conan config install/install-pkg, but it is worth to keep the configuration decoupled from the package recipes, it helps a lot to maintain recipes clean and to be able to build different configurations easily and in an scalable way (you can add configurations without having to modify every recipe)

[memsharded]

Adding it via --conf=tools.cmake.cmaketoolchain:user_toolchain+=my_user_toolchain.cmake works, but via conanfile.py def generate(self): tc = CMakeToolchain(self) toolchain_path = os.path.abspath(os.path.join(self.source_folder, "Toolchain.cmake")) self.conf_info.append("tools.cmake.cmaketoolchain:user_toolchain", toolchain_path) tc.generate() doesn't seem to work

We would need more information about this, like a minimal reproducible example.

The self.conf_info.append() needs to go to the package_info() method for example, not in the generate() method. It is an information for the consumers of the package, not for the package itself.

[miguelgcasado]

Hi, I managed to make it work adding self.conf.define("tools.cmake.cmaketoolchain:user_toolchain", [toolchain_path]) rifht before instantiating the toolchain with CMakeToolchain(self) on the generate() method. Thanks!

[memsharded]

I am afraid that it might be an unexpected and unsupported way to achieve it. Please check https://docs.conan.io/2/knowledge/guidelines.html

Settings and configuration (conf) are read-only in recipes: The settings and configuration cannot be defined or assigned values in recipes. Something like self.settings.compiler = "gcc" in recipes shouldn’t be done. That is undefined behavior and can crash at any time, or just be ignored. Settings and configuration can only be defined in profiles, in command line arguments or in the profile.py plugin.

The self.conf is not writeable within recipes, and this might eventually change and give you an error in a future conan version

If you'd like to inject this directly into the CMakeToolchain, the way to go would be the user_toolchain block, something like:

def generate(self):
    toolchain = CMakeToolchain(conanfile)
    toolchain.blocks["user_toolchain"].values["paths"] = ["myowntoolchain.cmake"]
    toolchain.generate()

More examples about this in https://docs.conan.io/2/reference/tools/cmake/cmaketoolchain.html#customizing-the-content-blocks.

Please try that and let me know. Thanks!

[miguelgcasado]

Sounds good, I tried this solution and works as well. Thanks for the support!

[memsharded]

Great, then I am closing the ticket as solved. Don't hesitate to open new tickets for any further question or issue. Thanks for your feedback!

[petermbauer]

Sorry for re-using the issue but i have the exact same problem. The toolchain file is part of the repo (not something i can influence) so i would need to set toolchain.blocks["user_toolchain"].values["paths"] to somewhere under the source_folder but in the generate() method, self.source_folder points to the build folder. A workaround could be to export the toolchain file and use self.recipe_folder but that's a bit clunky.

[memsharded]

Yeah @petermbauer not very clear what is the actual layout of files, how things look like, like the toolchain is in a Git repo you mean? And you want to create the package with that tooclhain? But isn't that toolchain necessary for dependencies, or just for that package? Is it for conan create or just for building a consumer of Conan packages?

The self.source_folder should still point to the folder with the sources, but yes, the toolchain must be exported with the recipe, as exports_sources, and it will end in self.source_folder. Not sure what is the flow there, might be better to detail it in a new ticket.

[petermbauer]

Correct, the toolchain file is in the repo with the recipe and the sources and it is not used anywhere else. I am using the scm attribute so i ended up exporting the toolchain file and using self.recipe_folder to reference it in the generate(). Thx!

[memsharded]

I am using the scm attribute so i ended up exporting the toolchain file and using self.recipe_folder to reference it in the generate().

Ok, still it sounds like exports_sources and being able to reference it from the source_folder should be possible, please don't hesitate to open a new ticket with details if you'd like to check it. Thanks for the feedback!