Using bazel-toolchain 5.1.0

[UebelAndre]

I'm trying to upgrade rules_foreign_cc to use bazel-toolchain 5.1.0 but do not know how to convert rbe_autoconfig to the modern equivalent (or what the modern equivalent is). Can someone provide a snippet of what is required?

[philwo]

Unfortunately we don't have anything here yet, as soon as we figure it out ourselves for Bazel, I'll follow up!

[davido]

Gerrit Code Review project has the same issue.

After bumping bazel-toolchain from 5.0.0 to 5.1.0 we are getting this error:

$ bazel build --config=remote --remote_instance_name=projects/api-project-164060093628/instances/default_instance headless
INFO: Invocation ID: 97ecd014-f2bb-4f96-b44a-53c041a7ac1b
INFO: Options provided by the client:
  Inherited 'common' options: --isatty=1 --terminal_columns=120
INFO: Reading rc options for 'build' from /home/davido/projects/gerrit2/.bazelrc:
  'build' options: --workspace_status_command=python3 ./tools/workspace_status.py --repository_cache=~/.gerritcodereview/bazel-cache/repository --action_env=PATH --disk_cache=~/.gerritcodereview/bazel-cache/cas --java_toolchain=//tools:error_prone_warnings_toolchain_java11 --incompatible_strict_action_env --announce_rc --experimental_worker_multiplex=false
INFO: Reading rc options for 'build' from /home/davido/.bazelrc:
  'build' options: --experimental_generate_json_trace_profile --experimental_profile_cpu_usage
INFO: Found applicable config definition build:remote in file /home/davido/projects/gerrit2/tools/remote-bazelrc: --jobs=200 --disk_cache= --host_javabase=@rbe_jdk11//java:jdk --javabase=@rbe_jdk11//java:jdk --crosstool_top=@rbe_jdk11//cc:toolchain --extra_toolchains=@rbe_jdk11//config:cc-toolchain --extra_execution_platforms=@rbe_jdk11//config:platform --host_platform=@rbe_jdk11//config:platform --platforms=@rbe_jdk11//config:platform --action_env=BAZEL_DO_NOT_DETECT_CPP_TOOLCHAIN=1 --define=EXECUTOR=remote --remote_cache=remotebuildexecution.googleapis.com --remote_executor=remotebuildexecution.googleapis.com --remote_timeout=3600 --google_default_credentials
ERROR: error loading package '': Label '@bazel_toolchains//rules:rbe_repo.bzl' is invalid because 'rules' is not a package; perhaps you meant to put the colon here: '@bazel_toolchains//:rules/rbe_repo.bzl'?
INFO: Elapsed time: 1.029s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded)

I also don't see any migration suggestion in their release notes site: https://github.com/bazelbuild/bazel-toolchains/releases/tag/v5.1.0

We are using this rbe_autoconfig:

# Creates a default toolchain config for RBE.
rbe_autoconfig(
    name = "rbe_jdk11",
    java_home = "/usr/lib/jvm/11.29.3-ca-jdk11.0.2/reduced",
    use_checked_in_confs = "Force",
)

and this doesn't work any more with Bazel 4.1.0.

[philwo]

After bumping bazel-toolchain from 5.0.0 to 5.1.0 we are getting this error:

Do you mean 4.0.0 to 5.1.0? Because from 5.0.0 to 5.1.0 should be no breaking change. However 4.0 and 5.x are completely different things and need a different setup and thus would explain the error message. In order to migrate, I'd personally start with reading the docs for bazel-toolchains 5.x and try to set it up from scratch - as rbe_autoconfig is completely gone, there is no migration where you just have to change a few attributes, it's really a completely different approach to generating and managing the configs.

I'll look into upgrading Bazel over the next weeks and can post our progress here. Maybe we can also provide a canonical container + configs for Bazel CI, similar to how rbe-ubuntu1604 and the checked-in configs in bazel-toolchains worked, but I can't promise that yet, I have to see how the new thing works myself first.

[davido]

Do you mean 4.0.0 to 5.1.0? Because from 5.0.0 to 5.1.0 should be no breaking change.

No. I mean what I said: 5.0.0 to 5.1.0.

We are currently using bazel-toochain 5.0.0 and all works just fine with Bazel 4.0.0:

http_archive(
    name = "bazel_toolchains",
    sha256 = "1adf7a8e9901287c644dcf9ca08dd8d67a69df94bedbd57a841490a84dc1e9ed",
    strip_prefix = "bazel-toolchains-5.0.0",
    urls = [
        "https://mirror.bazel.build/github.com/bazelbuild/bazel-toolchains/archive/v5.0.0.tar.gz",
        "https://github.com/bazelbuild/bazel-toolchains/archive/v5.0.0.tar.gz",
    ],
)

load("@bazel_toolchains//rules:rbe_repo.bzl", "rbe_autoconfig")

# Creates a default toolchain config for RBE.
rbe_autoconfig(
    name = "rbe_jdk11",
    java_home = "/usr/lib/jvm/11.29.3-ca-jdk11.0.2/reduced",
    use_checked_in_confs = "Force",
)

Switch bazel-toochain from 5.0.0 to 5.1.0: rien ne va plus!

Also, as I said I cannot just bump Bazel from 4.0.0 to 4.1.0: without breaking RBE CI verification.

[davido]

This is the CL: [1] from February 2021, where I bumped bazel-toolchain to 5.0.0 in Gerrit Code Review project.

[1] https://gerrit-review.googlesource.com/c/gerrit/+/296769

[davido]

In order to migrate, I'd personally start with reading the docs for bazel-toolchains 5.x and try to set it up from scratch - as rbe_autoconfig is completely gone, there is no migration where you just have to change a few attributes, it's really a completely different approach to generating and managing the configs.

I cannot figure out how to generate something that would replace rbe_jdk11 used by gerrit.

I tried this in bazel-toolchains@HEAD (27f2db256e54e5748ee1cd9485ccd0d5444bf1c6):

  $ ./rbe_configs_gen --bazel_version=4.1.0 --toolchain_container=l.gcr.io/google/rbe-ubuntu16-04:latest --output_src_root=/home/davido/projects/gerrit --output_config_path=configs/rbe --exec_os=linux --target_os=linux

And it's generating something that is different from what I am getting from checked in rbe_jdk11 configuration.

So that it really boils down to that question:

What parameter should I pass to rbe_configs_gen to generate something equivalent to checked in rbe_jdk11, but for Bazel 4.1.0 (and for newer Bazel releases for that matter)?

[davido]

What parameter should I pass to rbe_configs_gen to generate something equivalent to checked in rbe_jdk11, but for Bazel 4.1.0 (and for newer Bazel releases for that matter)?

This seems to work:

The command used in bazel-toochain project (v5.1.0):

  $ ./rbe_configs_gen \
  --bazel_version=4.1.0 \
  --toolchain_container=l.gcr.io/google/rbe-ubuntu18-04:latest \
  --output_src_root=/home/davido/projects/gerrit \
  --output_config_path=tools/rbe \
  --exec_os=linux \
  --target_os=linux

Given that there is no native support for JDK 11 in bazel-toolchain, custom jdk11 was added to generated tools/rbe/java/BUILD file:

java_runtime(
    name = "jdk11",
    srcs = [],
    java_home = "/usr/lib/jvm/11.29.3-ca-jdk11.0.2/reduced",
)

[philwo]

No. I mean what I said: 5.0.0 to 5.1.0.

Got it - thanks for confirming! That's really surprising, I thought that 5.0.0 had already removed these things and 5.1.0 was just a patch release with some bug fixes. I haven't investigated it too much though.

Thank you so much for documenting your steps and progress here! This will save others (including me 😄) a lot of time!