Suggestions on handling OS-specific activation?

[dhalperi]

We are looking at using graphviz-java, which has the following in its pom.xml: https://github.com/nidi3/graphviz-java/blob/master/graphviz-java/pom.xml#L128-L191

tl;dr: OS-specific dependencies are activated based on the operating system on which the build is processed. This seems to be making it into the maven_install.json file -- if I run on macOS, only the macOS dependency is added.

Has anyone run into this before? Presumably what I want is for all these to be added to the maven_install.json, and then have them be platform-dependent dependencies. Does anyone have advice for how to actually do this?

[thomasbao12]

I think I'm having a similar issue. orc-core 1.9.1 has a dependency on hadoop-client-api, but only under the java 17 profile. See https://repo1.maven.org/maven2/org/apache/orc/orc-core/1.9.1/orc-core-1.9.1.pom

However, for this maven_install target, I see hadoop-client-api in the maven_install.json

maven_install(name = "multiversion_maven_org_apache_orc_orc_core_nohive_1_9_1", artifacts = [maven.artifact("org.apache.orc", "orc-core", "1.9.1", packaging = "jar", classifier = "nohive")], maven_install_json = "@//bazel/workspace/third_party:multiversion_maven_org_apache_orc_orc_core_nohive_1_9_1_install.json", **kwargs)

This is what the install.json looks like:

"dependencies": {
    "org.apache.hadoop:hadoop-client-api": [
      "org.xerial.snappy:snappy-java"
    ],
    "org.apache.orc:orc-core:jar:nohive": [
      "io.airlift:aircompressor",
      "org.apache.commons:commons-lang3",
      "org.apache.orc:orc-shims",
      "org.jetbrains:annotations",
      "org.slf4j:slf4j-api",
      "org.threeten:threeten-extra"
    ],
    "org.apache.orc:orc-shims": [
      "org.apache.hadoop:hadoop-client-api",
      "org.slf4j:slf4j-api"
    ]
  },

[Silic0nS0ldier]

In v5 and v6 dependencies with activations do appear to be handled, at least at resolution time. As a result maven_install.json becomes platform-dependent (can be worked around with exclusions).

[mattnworb]

Presumably what I want is for all these to be added to the maven_install.json, and then have them be platform-dependent dependencies. Does anyone have advice for how to actually do this?

We've run into something similar recently with https://github.com/hyperxpro/Brotli4j/blob/e08da01d544bc63d3bfee17bae35db7ffeaf0eb3/brotli4j/pom.xml#L44-L214 and what is described here seems like the ideal outcome as well - the generated jvm_import shouldn't have a dep on just the dependency of the native platform based on which OS the pin command was run from:

❯ bazel query @maven//:com_aayushatharva_brotli4j_brotli4j --output=build
# /private/var/tmp/_bazel_mattbrown/24f8f5a0eeb6b27b916bd8a74f8ebe77/external/rules_jvm_external++maven+sp_maven/BUILD:21724:11
jvm_import(
  name = "com_aayushatharva_brotli4j_brotli4j",
  ...
  deps = ["@maven//:com_aayushatharva_brotli4j_native_osx_aarch64", "@maven//:com_aayushatharva_brotli4j_service"],
  ...
)

The issue is that coursier fetch is handling activating profiles within a pom based on the OS etc. coursier fetch supports passing --profile to explicitly activate profiles by name, so

maven.install(
    name = "...",
    additional_coursier_options = [
        "--profile",
        "osx-aarch64",
        "--profile",
        "linux-x86_64",
        ...
    ]
)

at least results in the generated jvm_import having additional deps:

jvm_import(
  name = "com_aayushatharva_brotli4j_brotli4j",
  ...
  deps = ["@maven//:com_aayushatharva_brotli4j_native_linux_x86_64", "@maven//:com_aayushatharva_brotli4j_native_osx_aarch64", "@maven//:com_aayushatharva_brotli4j_service"],
   ...
)

but it seems to me like what would really be wanted is to have a select() in there to choose the dep based on the OS, but I don't think there is really a way for RJE to know which dependencies in the output from coursier were from OS-specific activations (and of course activating profiles by name is brittle, different pom.xmls won't necessarily use the same profile name etc).

[shs96c]

This is something that will need to be solved in the underlying resolvers. By default we use coursier, but there are also resolvers based on maven and gradle now.

We can look at passing the profiles to coursier, which should get us quite a long way, though @mattnworb is right that the ideal fix would involve a select