rules_go

rules_go copied to clipboard

Go toolchain checks which linker flags are available by running the linker on a temporary file. Before Go 1.21, this command was run in a temporary directory, but this has now been fixed upstream. https://github.com/golang/go/issues/59952

This change will not be needed for Go versions 1.21 and above, but will be beneficial for users on lower versions and using a hermetic toolchain.

See https://github.com/grailbio/bazel-toolchain/issues/183 for some more context.

I did not do any performance regression testing. But some performance hit may be expected. Go toolchain is now running a small linker command before running the actual link command. Previously, that small linker command was failing trivially.

I did not do any performance regression testing. But some performance hit may be expected. Go toolchain is now running a small linker command before running the actual link command. Previously, that small linker command was failing trivially.

Our CI is experiencing a 2x overall build time (without cache) with this change applied, compared to without this change.

I would like to do a little more investigation, and see if there's a way we can fix this issue without adding latency to the build. Otherwise, we should guard this feature behind a config flag so that users are not impacted unless they need this feature.

Which hermetic toolchain do you use in your CI? Maybe I can help with this investigation if I find time.

It is also very likely that you would hit the increased build time issue when you move to Go 1.21, which has a similar change as in this PR, but built into the toolchain itself.

It is also very likely that you would hit the increased build time issue when you move to Go 1.21, which has a similar change as in this PR, but built into the toolchain itself.

I tested it with Go 1.21.1

Which hermetic toolchain do you use in your CI? Maybe I can help with this investigation if I find time.

We use zig_toolchain at version 0.11.0

http_archive(
    name = "hermetic_cc_toolchain",
    sha256 = "28fc71b9b3191c312ee83faa1dc65b38eb70c3a57740368f7e7c7a49bedf3106",
    urls = [
        "https://github.com/uber/hermetic_cc_toolchain/releases/download/{0}/hermetic_cc_toolchain-{0}.tar.gz".format(HERMETIC_CC_TOOLCHAIN_VERSION),
    ],
)

load("@hermetic_cc_toolchain//toolchain:defs.bzl", zig_toolchains = "toolchains")

zig_version = "0.11.0"
zig_toolchains(
    host_platform_sha256 = {
        "linux-aarch64": "956eb095d8ba44ac6ebd27f7c9956e47d92937c103bf754745d0a39cdaa5d4c6",
        "linux-x86_64": "2d00e789fec4f71790a6e7bf83ff91d564943c5ee843c5fd966efc474b423047",
        "macos-aarch64": "c6ebf927bb13a707d74267474a9f553274e64906fd21bf1c75a20bde8cadf7b2",
        "macos-x86_64": "1c1c6b9a906b42baae73656e24e108fd8444bb50b6e8fd03e9e7a3f8b5f05686",
    },
    url_formats = [
        "https://ziglang.org/builds/zig-{host_platform}-{version}.{_ext}",
    ],
    version = zig_version,
)

I tried to reproduce with bazel build --incompatible_enable_cc_toolchain_resolution --extra_toolchains @zig_sdk//toolchain:linux_amd64_gnu.2.34 --remote_cache= --disk_cache= //tests/core/cgo:c_srcs and the updated toolchain you posted above.

I can not reproduce what you are describing; maybe I am not calling with the right toolchain target. In the example above, the link step took 20 seconds on first invocation (clean /tmp dir), and then 4 seconds for each repeated run (with bazel clean in between). This is both with and without this change.

The issue in your CI could be more specific to the setup you have.

Closing this because it has been roughly a year since the release of Go 1.21, so it is less needed now. The extra diligence for regression testing makes it not worthwhile.