[bug] When compiler is clang-cl and runtime=None, incorrect architecture flags are generated

[mattarroz]

Describe the bug

Conan assumes that runtime=None implies that msys2 clang is used (see here). However, in our project we need runtime to be None but still use clang-cl to cross compile from Linux to Windows, where -m32 is an incorrect flag.

=== As a fix, I suggest adding a frontend setting to the compiler section of the settings.yml file:

    clang:
        version: ["3.3", "3.4", "3.5", "3.6", "3.7", "3.8", "3.9", "4.0",
                  "5.0", "6.0", "7.0", "7.1",
                  "8", "9", "10", "11", "12", "13", "14", "15", "16", "17"]
        libcxx: [None, libstdc++, libstdc++11, libc++, c++_shared, c++_static]
        cppstd: [None, 98, gnu98, 11, gnu11, 14, gnu14, 17, gnu17, 20, gnu20, 23, gnu23]
        runtime: [None, MD, MT, MTd, MDd, static, dynamic]
        runtime_type: [None, Debug, Release]
        runtime_version: [None, v140, v141, v142, v143]
        frontend: [clang, msvc]

Then, this setting is used to explicitly check for the msvc frontend variant instead of implicitly making the assumption runtime=None => frontend=clang.

E.g.

        if compiler_frontend == "msvc":
            return ""

instead of

        if runtime is not None:
            return ""

This setting can then be used other places as well to explicitly check for clang-cl instead of implicit logic such as runtime=None. We already tested this change successfully. If you think this is a good idea, I can create a corresponding PR.

How to reproduce it

I've prepared Autotools and CMake examples in a small repository. For Autotools, -m64 -g is generated, for CMake (link) just -m64 in terms of incorrect (?) flags. There is a Dockerfile for complete build environment reproduction. The conan create logfiles show the complete session. The rest of the build process cannot be disclosed, however I hope the problem is clear already.

[memsharded]

Hi @mattarroz

Thanks for your feedback.

Conan assumes that runtime=None implies that msys2 clang is used (see here). However, in our project we need runtime to be None but still use clang-cl to cross compile from Linux to Windows, where -m32 is an incorrect flag.

I'd like to understand this a bit better. What clang are you using? If it is not the msys2 clang, isn't it the LLVM/Clang?

Not sure if you have read it, but https://blog.conan.io/2022/10/13/Different-flavors-Clang-compiler-Windows.html explains a bit in depth the different variants and how Conan models them.

[mattarroz]

We are using LLVM/clang through the clang-cl command line interface. BTW, another incorrect flag generated by conan is the -g flag:

builder@debian:/tmp$ clang-cl -g test.c /c
clang: warning: unknown argument ignored in clang-cl: '-g' [-Wunknown-argument]
builder@debian:/tmp$ clang-cl -v
Debian clang version 14.0.6
Target: x86_64-pc-windows-msvc
Thread model: posix
InstalledDir: /usr/bin
builder@debian:/tmp

This flag is generated by conan here.

Here, we cannot assume from the fact that compiler is "clang" that the flag -g is understood. Again, I think the "cl.exe" frontend of clang needs to be distinguished from the standard clang/gnu style frontend (e.g. frontend: [clang, msvc] in settings.yml)

[memsharded]

We are using LLVM/clang through the clang-cl command line interface.

But that LLVM/Clang does use the msvc runtime. Even when it is cross-building, when it is cross-building for Windows, it requires the msvc headers and libs somewhere, even the windows sdk, and command line invocations args include something in the line of:

-target x86_64-pc-windows-msvc -L/opt/win-sdk/10.0.22621/crt/lib/x86_64 -L/opt/win-sdk/10.0.22621/sdk/lib/ucrt/x86_64

This means that the compiler.runtime for msvc is still used, as described in the blog post above, those are the options for Windows, using the msvc backend or the msys2 backend.

I know about the frontend issue, it is true that this is something pending, there is no explicit configuration to decide between the clang and the clang-cl frontends, (both using the msvc backend). We should follow up about that in another ticket (there might be other ticket for this already open), but lets try to keep this ticket focused on the runtime=None. Can we please step back a little and use compiler.runtime=dynamic and then see what would be failing?

The best is always to work with things we can reproduce, for example a simple project created with conan new cmake_lib -d name=mypkg -d version=0.1. For the compiler setup, docker images typically help a lot too.

[mattarroz]

But that LLVM/Clang does use the msvc runtime. Even when it is cross-building, when it is cross-building for Windows, it requires the msvc headers and libs somewhere, even the windows sdk, [...]

We use the /nodefaultlib flag (see documentation) together with our custom C runtime library implementation. So one might as well argue that a runtime=costum setting would be correct instead of None. Anyway, the msvc runtime is definitely not linked.

I know about the frontend issue, it is true that this is something pending, there is no explicit configuration to decide between the clang and the clang-cl frontends, (both using the msvc backend). We should follow up about that in another ticket (there might be other ticket for this already open), but lets try to keep this ticket focused on the runtime=None. Can we please step back a little and use compiler.runtime=dynamic and then see what would be failing?

I agree, let's keep it focused without forgetting about the general picture. runtime=dynamic, generating a -m32 or -m64 flag, would not cause any concrete failure since clang-cl options can start with either / or - (as opposed to -g, which is ignored with a warning as I demonstrated before). However, the logic of this code block (and the comments in the code) implies that there aren't any flags like -m32 supposed to be passed to clang-cl, this is why I believe there is something wrong here.

If you want, we can move the discussion to another ticket with a more generic scope.

The best is always to work with things we can reproduce, for example a simple project created with conan new cmake_lib -d name=mypkg -d version=0.1. For the compiler setup, docker images typically help a lot too.

For sure, I agree too. I would have provided such an example, however in this case I believe the problem can be understood from the conan source code and the settings.yml file. For the -g I could provide such an example if you want.

[memsharded]

We use the /nodefaultlib flag (see documentation) together with our custom C runtime library implementation. So one might as well argue that a runtime=costum setting would be correct instead of None. Anyway, the msvc runtime is definitely not linked.

Oh, very interesting, thanks for the explanation.

Then maybe adding a custom settings.yml with something like:

runtime: [custom, static, dynamic]

could make more sense, and it will not define the -m32/-m64 flags. But I think it will break the CMakeToolchain block defining the CMAKE_MSVC_RUNTIME_LIBRARY, which will require some fixes (I think those are doable, no big deal)

I am having a look to the code that references compiler.runtime and it is true that there are some assumptions there that might not hold for your current setup. I am trying to understand and think what would be the possibilities, I think we can keep investigating this in this thread.

Indeed, it would be good to have more clear goals, for example, the -g is defined in build_type_flags() which seems it only applies to build systems like autotools, but not cmake. So having something fully detailed and reproducible would be indeed great, are you really using CMake? Using CMakeToolchain?

The priority would be to do the necessary changes to allow the Conan toolchain generators to not add undesired flags.

Many thanks for your feedback.

[mattarroz]

Indeed, it would be good to have more clear goals, for example, the -g is defined in build_type_flags() which seems it only applies to build systems like autotools, but not cmake. So having something fully detailed and reproducible would be indeed great, are you really using CMake? Using CMakeToolchain?

We are using conancenter recipes, so Autotools and CMake are used. I've prepared Autotools and CMake examples in a small repository. For Autotools, -m64 -g is generated, for CMake (link) just -m64 in terms of incorrect (?) flags. There is a Dockerfile for complete build environment reproduction. The conan create logfiles show the complete session. The rest of the build process cannot be disclosed, however I hope the problem is clear already.

Thank you for collaborating :-)

[waterwalker1429]

Mattarroz reported that conan's CMakeToolchain and AutotoolsToolchain generate compiler flags that are not compliant with the clang-cl command line (e.g. -g, -m64). I would like to add that also linker library paths, linker flags, and include paths are not compliant with the clang-cl command line when using conan's AutotoolsDeps generator. To illustrate this, I forked mattarroz's example repo and extended the autotools example to show the AutotoolsDeps related problem. Addtionally, I have added a detailed instruction on how to reproduce the issue, as well as a log file indicating the observed behavior. Please consider solving these clang-cl related issues together to fully enable a clang-cl based conan workflow.

[memsharded]

I am not sure if it has been mentioned before. It is known that AutotoolsDeps has some limitations for Windows usage: https://docs.conan.io/2/examples/tools/autotools/create_your_first_package_windows.html

This wouldn't be unique to clang-cl, it is an AutotoolsDeps limitation

Thanks all for the contributed projects code, that will really help.

[memsharded]

I am doing PR https://github.com/conan-io/conan/pull/17387, trying for the next release.

This PR uses conf tools.build:compiler_executables = {'c': 'clang-cl', ...} as information to decide what to do, both for:

• Architecture flag
• BuildType flag (-g and other optimization flags)

I think that a new setting might not be necessary, as the idea is that the binaries are binary compatible, and only the frontend and how flags are managed is the difference.

Please let me know what you think, and if you can give it a try, running from my source branch from the PR, that would also be very valuable feedback. Thanks!

[memsharded]

This has been closed by https://github.com/conan-io/conan/pull/17387, for next Conan 2.10. Looking forward your feedback, thanks!

[mattarroz]

This has been closed by #17387, for next Conan 2.10. Looking forward your feedback, thanks!

Sorry for missing this! I prefer your solution—it avoids the need for an additional (and redundant) setting 👍

[memsharded]

Great, thanks for the feedback!