Pass `--host_jvmopt` to host options attribute

[ThomasCJY]

Background We hit a similar issue with https://github.com/bazelbuild/bazel/issues/12403 when trying to add --host_jvmopt in our build. The error message looks like this:

ERROR: file 'external/remote_java_tools/proguard' is generated by these conflicting actions:
Label: @remote_java_tools//:proguard
RuleClass: java_binary rule
Configuration: a46bb511e54d3417610f91a9e0438015d7a78d7ac4f86f7c2440dea289d49498, 88e50b932f2f46570420b692a53eb24e621234bce56b60adfcb78e0fc18f0b30
Mnemonic: TemplateExpand
Action key: 8f032555b33813e37fe81cc562b797ada9bb13f67fa7f56de2561b90dea164f6, 38d3af6a15195200b199160dca175890cd7d2b77f291e8ec0cd4330ebe825fee
Progress message: Expanding template external/remote_java_tools/proguard
PrimaryInput: (null)
PrimaryOutput: File:[[<execution_root>]bazel-out/darwin_arm64-opt-exec-2B5CBBC6/bin]external/remote_java_tools/proguard
Owner information: ConfiguredTargetKey{label=@remote_java_tools//:proguard, config=BuildConfigurationValue.Key[a46bb511e54d3417610f91a9e0438015d7a78d7ac4f86f7c2440dea289d49498]}, ConfiguredTargetKey{label=@remote_java_tools//:proguard, config=BuildConfigurationValue.Key[88e50b932f2f46570420b692a53eb24e621234bce56b60adfcb78e0fc18f0b30]}
MandatoryInputs: are equal
Outputs: are equal 

Change Follow the idea of this fix (https://github.com/bazelbuild/bazel/commit/e6670825b1e183f81f5c864aafd425d512fa9ff5), we tried adding hostJvmOpts to the host options attribute and we verified it fixed the issue.

[nkoroste]

@ahumesky

[ThomasCJY]

@lberki ptal

[ThomasCJY]

looks like @lberki is not available. @sgowroji anyone else can take a look at this change?

[gregestren]

One diagnostic followup is to see the exact difference in the configurations expressed. So given:

Configuration: a46bb511e54d3417610f91a9e0438015d7a78d7ac4f86f7c2440dea289d49498, 88e50b932f2f46570420b692a53eb24e621234bce56b60adfcb78e0fc18f0b30

what does

$ bazel config a46bb 88e50

show? (those hashes are really long so you can pass a simple prefix to save typing)

You could also try passing --experimental_output_directory_naming_scheme=diff_against_baseline, which is a promising new feature that makes conflicting action errors basically impossible.

[lukaciko]

We got the same conflicting action errors in our Android build with Bazel 6.x when trying to run with --host_jvmopt="-XX:-MaxFDLimit".

bazelisk config ba4daa bfc79f gives us:

Displaying diff between configs ba4daa and bfc79fa751b
FragmentOptions com.google.devtools.build.lib.rules.java.JavaOptions {
  jvmopt: [-XX:ErrorFile=/dev/stderr], [-XX:-MaxFDLimit]
}

so nothing too surprising there.

Setting --experimental_output_directory_naming_scheme=diff_against_baseline resolves the issue for us and I see that's now the default setting on the master branch. It seems like a big change, so we might go with something more conservative while we're on 6.x.

[fmeum]

@gregestren Even though this is somewhat mitigated by --experimental_output_directory_naming_scheme=diff_against_baseline, this still looks like a bug to me. Did you close this on purpose or would you consider merging?

[gregestren]

I think I closed this because it lingered without an https://github.com/bazelbuild/bazel/labels/awaiting-user-response update. So it became orphaned.

Happy to re-review given these last comments.

[BalestraPatrick]

@gregestren I fixed the conflicts and it's ready for another review.

[gregestren]

Could https://github.com/bazelbuild/bazel/pull/18561 be a candidate solution for 6.4.0?

It seems like a big change

It and diff_against_baseline are definitely bigger impact changes than this. But they're intended to be strict improvements that ask nothing of the user. And they've both gotten a good amount of repo testing to vet their safety.

@fmeum when you say "somewhat mitigated", which scenarios are you thinking of that still crash?

I'm pushing back some because diff_against_dynamic_baseline is intended to more principally address the source of these errors in one fell swoop. It'd be nice if we could just eliminate the class of errors vs. having to remember to pass another --host_* flag every time the issue pops up.

[fmeum]

At this point I am quite convinced that diff_against_dynamic_baseline is safe from the point of view of correctness and performance, but I still fear path length issues on Windows and thus am not 100% convinced that it is a safe cherry pick.

@fmeum when you say "somewhat mitigated", which scenarios are you thinking of that still crash?

As far as I understand the situation, diff_against_dynamic_baseline would definitely mitigate the crash in this case. What I am worried about is correctness: Could there be a chain of transitions that results in different JVM options (host or target) due to "losing" the host options when applying the exec transition? Happy to hear from you that this isn't actually possible though :-)

[gregestren]

At this point I am quite convinced that diff_against_dynamic_baseline is safe from the point of view of correctness and performance, but I still fear path length issues on Windows and thus am not 100% convinced that it is a safe cherry pick.

To be clear, you still support https://github.com/bazelbuild/bazel/pull/18561 for 7.0? @sdtwigg is intending to merge.

@fmeum when you say "somewhat mitigated", which scenarios are you thinking of that still crash?

As far as I understand the situation, diff_against_dynamic_baseline would definitely mitigate the crash in this case. What I am worried about is correctness: Could there be a chain of transitions that results in different JVM options (host or target) due to "losing" the host options when applying the exec transition? Happy to hear from you that this isn't actually possible though :-)

@sdtwigg anything you could think of?