POM with binary type dependencies

[jarreds]

The POM for DynamoDB local com.amazonaws:DynamoDBLocal:1.11.477 in https://s3-us-west-2.amazonaws.com/dynamodb-local/release includes some binary dependencies:

    <dependency>
      <groupId>com.almworks.sqlite4java</groupId>
      <artifactId>libsqlite4java-linux-amd64</artifactId>
      <version>${sqlite4java.version}</version>
      <type>so</type>
      <scope>runtime</scope>
    </dependency>
    <dependency>
      <groupId>com.almworks.sqlite4java</groupId>
      <artifactId>sqlite4java-win32-x64</artifactId>
      <version>${sqlite4java.version}</version>
      <type>dll</type>
      <scope>runtime</scope>
    </dependency>
    <dependency>
      <groupId>com.almworks.sqlite4java</groupId>
      <artifactId>sqlite4java-win32-x86</artifactId>
      <version>${sqlite4java.version}</version>
      <type>dll</type>
      <scope>runtime</scope>
    </dependency>
    <dependency>
      <groupId>com.almworks.sqlite4java</groupId>
      <artifactId>libsqlite4java-osx</artifactId>
      <version>${sqlite4java.version}</version>
      <type>dylib</type>
      <scope>runtime</scope>
    </dependency>

So we get:

The artifact for com.almworks.sqlite4java:libsqlite4java-linux-amd64:so:1.0.392 was not downloaded. Perhaps its packaging type is
not one of: jar,aar,bundle,eclipse-plugin?

Are there any suggestions for handling transitive dependencies of this sort?

[jarreds]

Noting that I'm able to work around this by using exclusions + http_file + resources in the java_binary. It would be nice if the the rules in this repo could handle it automagically though. Not sure if that's possible.

# WORKSPACE

http_file(
    name = "com_almworks_sqlite4java_libsqlite4java_linux_amd64",
    downloaded_file_path = "libsqlite4java-linux-amd64-1.0.392.so",
    urls = ["https://repo1.maven.org/maven2/com/almworks/sqlite4java/libsqlite4java-linux-amd64/1.0.392/libsqlite4java-linux-amd64-1.0.392.so"],
)

http_file(
    name = "com_almworks_sqlite4java_libsqlite4java_osx",
    downloaded_file_path = "libsqlite4java-osx-1.0.392.dylib",
    urls = ["https://repo1.maven.org/maven2/com/almworks/sqlite4java/libsqlite4java-osx/1.0.392/libsqlite4java-osx-1.0.392.dylib"],
)

http_file(
    name = "org_eclipse_jetty_orbit_javax_servlet",
    downloaded_file_path = "javax.servlet-3.0.0.v201112011016.jar",
    urls = ["https://repo1.maven.org/maven2/org/eclipse/jetty/orbit/javax.servlet/3.0.0.v201112011016/javax.servlet-3.0.0.v201112011016.jar"],
)

maven_install(
    artifacts = [
        maven.artifact(
            group = "com.amazonaws",
            artifact = "DynamoDBLocal",
            version = "1.11.477",
            exclusions = [
                "com.almworks.sqlite4java:libsqlite4java-linux-amd64",
                "com.almworks.sqlite4java:libsqlite4java-linux-i386",
                "com.almworks.sqlite4java:libsqlite4java-osx",
                "com.almworks.sqlite4java:sqlite4java-win32-x64",
                "com.almworks.sqlite4java:sqlite4java-win32-x86",
                "org.eclipse.jetty.orbit:javax.servlet",
            ],
        ),
    ],
)

# BUILD

java_import(
    name = "org_eclipse_jetty_orbit_javax_servlet",
    jars = [
        "@org_eclipse_jetty_orbit_javax_servlet//file",
    ],
)

java_binary(
    name = "dynamodb-local",
    args = [
        "-sharedDb",
        "-inMemory",
    ],
    data = [
        "@com_almworks_sqlite4java_libsqlite4java_linux_amd64//file",
        "@com_almworks_sqlite4java_libsqlite4java_osx//file",
    ],
    jvm_flags = [
        "-Djava.library.path=$$JAVA_RUNFILES/com_almworks_sqlite4java_libsqlite4java_osx/file/:$$JAVA_RUNFILES/com_almworks_sqlite4java_libsqlite4java_linux_amd64/file/",
    ],
    main_class = "com.amazonaws.services.dynamodbv2.local.main.ServerRunner",
    visibility = ["//visibility:public"],
    runtime_deps = [
        "org_eclipse_jetty_orbit_javax_servlet",
        "@maven//:com_almworks_sqlite4java_sqlite4java",
        "@maven//:com_amazonaws_DynamoDBLocal",
    ],
)

[jin]

Marking this as a feature request and thanks for providing the clear workaround.

FWIW, on 2.0.1 I can include the "com.amazonaws:DynamoDBLocal:1.11.477" artifact in maven_install and not see a crash. The binaries aren't downloaded so you don't need to write the exclusions in the spec.

[joprice]

@jin I tried the same and get errors like

in deps attribute of jvm_import rule @scala_2_11//:com_amazonaws_DynamoDBLocal: rule '@scala_2_11//:com_almworks_sqlite4java_libsqlite4java_linux_i386_so' does not exist"

when building. I had to use a similar workaround as above.

[daicoden]

log4j exports some test helpers with a packaging as test-jar. Packaging doesn't seem to affect which artifact is actually retrieved. (I think same root issue)

[dsilvasc]

Maybe the same problem for com.oracle.substratevm:svm:19.2.0.1:

external/maven/BUILD:509:11: in deps attribute of jvm_import rule @maven//:org_graalvm_truffle_truffle_nfi: rule '@maven//:org_graalvm_truffle_truffle_nfi_native_darwin_amd64_tar_gz' does not exist
external/maven/BUILD:509:11: in deps attribute of jvm_import rule @maven//:org_graalvm_truffle_truffle_nfi: rule '@maven//:org_graalvm_truffle_truffle_nfi_native_linux_amd64_tar_gz' does not exist
external/maven/BUILD:509:11: in deps attribute of jvm_import rule @maven//:org_graalvm_truffle_truffle_nfi: rule '@maven//:org_graalvm_truffle_truffle_nfi_native_linux_aarch64_tar_gz' does not exist

[purkhusid]

Hitting the exact same issue with the same dependency. So far this has been the only issue I've had with rules_jvm_external.

[checorone]

Same for quarkus @maven//:io_quarkus_quarkus_grpc_codegen: rule '@maven//:com_google_protobuf_protoc_exe_windows_x86_32' does not exist. It expects JAR, but file have .exe extension...

[dzbarsky]

https://github.com/bazelbuild/rules_jvm_external/pull/959/files is not fully automagical but a bit nicer experience