buck2 icon indicating copy to clipboard operation
buck2 copied to clipboard

Support multiple execution platforms with system_cxx_toolchain

Open Overhatted opened this issue 9 months ago • 0 comments

As mentioned in #612, I have a C++ project that can be built for either Linux or Windows, without any cross-compilation so Linux and Windows are also the two execution platforms. Since there is remote execution, I wanted to be able to build for Linux from my Windows machine using RE, and vice versa.

From the response I got in the above issue, I need to add an exec_dep to the toolchain, pointing a compiler target, that then has a target_compatible_with with the execution platform the compiler can run on.

I tried to add the target_compatible_with but it didn't work, probably intentional.

So I moved some parts of the current system_cxx_toolchain to a separate NativeCompiler provider that is then returned by some rules defining a compiler. This compiler target then has the target_compatible_with attribute.

Can this be merged or should I just have this toolchain in my own repository? It seems useful for everyone, the only disadvantage I can see being that it is not fully backwards compatible. Hopefully I didn't completely misunderstand how execution platforms are supposed to be used.

I improved the Visual Studio example to use this new updated toolchain to easily support remote execution.

Manual testing

I used the Visual Studio example and setup my RE workers with Buildbarn. Unfortunately its Windows workers don't really work and the linking step fails with "The system cannot open the device or file specified.". I haven't investigated why.

Local builds

Windows

buck2 build --target-platforms //buck2_utils/platforms:windows_debug :main

Passes

Linux

buck2 build --target-platforms //buck2_utils/platforms:linux_debug :main

Passes

Remote builds

From Windows

Target Windows
buck2 build --no-remote-cache -c buck2_re_client.enabled=true --target-platforms //buck2_utils/platforms:windows_debug :main

Linking fails (The system cannot open the device or file specified.)

Target Linux
buck2 build --no-remote-cache -c buck2_re_client.enabled=true --target-platforms //buck2_utils/platforms:linux_debug :main

Passes

From Linux

Target Windows
buck2 build --no-remote-cache -c buck2_re_client.enabled=true --target-platforms //buck2_utils/platforms:windows_debug :main

Linking fails (The system cannot open the device or file specified.)

Target Linux
buck2 build --no-remote-cache -c buck2_re_client.enabled=true --target-platforms //buck2_utils/platforms:linux_debug :main

Passes

Overhatted avatar Apr 28 '24 21:04 Overhatted