apache
circe-checksum, cpu-affinity and native-io contain invalid maven metadata in pom.xml for packaging
BUG REPORT
Describe the bug
circe-checksum, cpu-affinity and native-io contain invalid maven metadata in pom.xml for packaging. This has an impact on Gradle builds. For example when using the Shadow plugin in Gradle, it won't properly locate the dependencies.
To Reproduce
Steps to reproduce the behavior:
- Go to https://repo1.maven.org/maven2/org/apache/bookkeeper/circe-checksum/4.15.1/circe-checksum-4.15.1.pom
- You can see that
<packaging>nar</packaging>is present.
Expected behavior
jar should be the default type for public libraries. nar is an exceptional case.
Workaround for Gradle builds
Placing this in build.gradle resolves the issue
// Workaround for invalid metadata for Bookkeeper dependencies which contain <packaging>nar</packaging> in pom.xml
configurations.all {
resolutionStrategy.eachDependency { DependencyResolveDetails details ->
if (details.requested.group == 'org.apache.bookkeeper' &&
details.requested.name in ['circe-checksum', 'cpu-affinity', 'native-io']) {
details.artifactSelection { ArtifactSelectionDetails asDetails ->
asDetails.selectArtifact('jar', null, null)
}
}
}
}
or build.gradle.kts
configurations {
all {
resolutionStrategy {
eachDependency {
if (requested.group == "org.apache.bookkeeper" &&
requested.name in listOf("circe-checksum", "cpu-affinity", "native-io")
) {
// Workaround for invalid metadata for Bookkeeper dependencies which contain
// <packaging>nar</packaging> in pom.xml
artifactSelection {
selectArtifact("jar", null, null)
}
}
}
}
}
(in a sample build)
Additional context
Here's a sample error message that results of the invalid pom.xml metadata for an application that uses pulsar-client-original dependency and uses Shadow plugin to build a fat jar file:
[2022-09-30 10:31:57,947] [pulsar-client-io-1-1] [org.apache.pulsar.client.impl.ProducerImpl] WARN [test] [standalone-0-11] error while create opSendMsg by batch message container
java.lang.NoClassDefFoundError: com/scurrilous/circe/checksum/Crc32cIntChecksum
at org.apache.pulsar.common.protocol.Commands.serializeCommandSendWithSize(Commands.java:1519) ~[sample-pulsar-gradle-all.jar:?]
at org.apache.pulsar.common.protocol.Commands.newSend(Commands.java:540) ~[sample-pulsar-gradle-all.jar:?]
at org.apache.pulsar.common.protocol.Commands.newSend(Commands.java:507) ~[sample-pulsar-gradle-all.jar:?]
at org.apache.pulsar.client.impl.ProducerImpl.sendMessage(ProducerImpl.java:771) ~[sample-pulsar-gradle-all.jar:?]
at org.apache.pulsar.client.impl.BatchMessageContainerImpl.createOpSendMsg(BatchMessageContainerImpl.java:200) ~[sample-pulsar-gradle-all.jar:?]
at org.apache.pulsar.client.impl.ProducerImpl.batchMessageAndSend(ProducerImpl.java:2002) ~[sample-pulsar-gradle-all.jar:?]
at org.apache.pulsar.client.impl.ProducerImpl.lambda$connectionOpened$14(ProducerImpl.java:1631) ~[sample-pulsar-gradle-all.jar:?]
at org.apache.pulsar.common.util.Runnables$CatchingAndLoggingRunnable.run(Runnables.java:54) ~[sample-pulsar-gradle-all.jar:?]
at io.netty.util.concurrent.PromiseTask.runTask(PromiseTask.java:98) ~[sample-pulsar-gradle-all.jar:?]
at io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:176) ~[sample-pulsar-gradle-all.jar:?]
at io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:174) ~[sample-pulsar-gradle-all.jar:?]
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:167) ~[sample-pulsar-gradle-all.jar:?]
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:470) ~[sample-pulsar-gradle-all.jar:?]
at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:394) ~[sample-pulsar-gradle-all.jar:?]
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:995) ~[sample-pulsar-gradle-all.jar:?]
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[sample-pulsar-gradle-all.jar:?]
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[sample-pulsar-gradle-all.jar:?]
at java.lang.Thread.run(Thread.java:833) ~[?:?]
Caused by: java.lang.ClassNotFoundException: com.scurrilous.circe.checksum.Crc32cIntChecksum
... 18 more
I've encountered this bug when running a client that has a transitive dependency on org.apache.bookkeeper::circe-checksum and built with Gradle. When run in a Docker container that has a base image of openjdk:11 or openjdk:17, I get the same NoClassDefFoundError: com/scurrilous/circe/checksum/Crc32cIntChecksum failure.
I experimented with changing the packaging in the POM to jar, installing the library in my local m2 repo, and rebuilding my client app. When the transitive dep is a jar, the class loader finds Crc32cIntChecksum and I get the fall back message:
WARN com.scurrilous.circe.checksum.Crc32cIntChecksum - Failed to load Circe JNI library. Falling back to Java based CRC32c provider
The Gradle build is creating a fat jar for my client app. If it includes circe-checksum as a .nar, then the class loader doesn't find the class when running in the container, but it does when the build includes circe-checksum as a .jar.
For reasons I cannot explain, when running my client app with Gradle run on macOS, I get the fall back message with the .nar.
I encountered the class loader error with 4.14.4, 4.14.5, and 4.15.1.
Now we have switched to maven, I think we can close this issue now.
This isn't about using maven or gradle in bookkeeper.
@lhotari What can we do with dealing this issue? In nar-maven-plugin's documentation[1], <packaging>nar<packaging> is required. Adding some docs to gradle user?
[1] https://maven-nar.github.io/usage.html
@lhotari What can we do with dealing this issue? In
nar-maven-plugin's documentation[1],<packaging>nar<packaging>is required. Adding some docs to gradle user?[1] https://maven-nar.github.io/usage.html
I'd assume that it's possible to preprocess the pom before publishing? The packaging should be replaced before publishing.
