maven-shade-plugin
maven-shade-plugin copied to clipboard
apache
[MSHADE-156] shade plugin is transforming also strings that are not supposed to be transformed
Neeme Praks opened MSHADE-156 and commented
I'm shading several third party libraries into a single JAR. In the context of this issue, there are two important libraries:
- com.nothome:javaxdelta:2.0.1 - this library is in "at.spardat.xma.xdelta" package.
- ch.qos.logback:logback-classic:1.0.11 - in this library, in class ch.qos.logback.classic.spi.StackTraceElementProxy, there is a method to convert stack trace elements into strings:
public String getSTEAsString() {
if (steAsString == null) {
steAsString = "at " + ste.toString();
}
return steAsString;
}
I use "org.myorgname.appname.shaded" package for shading.
During shading, the constant "at " is replaced with "org.myorgname.appname.shaded.at" -- shade plugin thinks that the "at" in the beginning of that string is a package name and shades it.
This results in an unfortunate side-effect: all logged stack traces now look like this:
Caused by: java.util.zip.ZipException: error in opening zip file
org.myorgname.appname.shaded.at java.util.zip.ZipFile.open(Native Method) ~[na:1.6.0_19]
org.myorgname.appname.shaded.at java.util.zip.ZipFile.<init>(ZipFile.java:114) ~[na:1.6.0_19]
org.myorgname.appname.shaded.at java.util.zip.ZipFile.<init>(ZipFile.java:131) ~[na:1.6.0_19]
Possible fixes:
- instead of just checking the first part of the package (at), check for the full package (at.spardat.xma.xdelta).
- allow to configure shade plugin to ignore certain string constants in certain classes (this would solve also some other issues we have with shade plugin being too eager).
- both of the above.
Affects: 2.1
Issue Links:
- SLING-8092 Relocation in Content Extension too bold ("causes")
- FLINK-13044 Shading of AWS SDK in flink-s3-fs-hadoop results in ClassNotFoundExceptions ("breaks")
Remote Links:
9 votes, 15 watchers
Jon McLean commented
I have a similar issue. In my case the incorrect transform causes runtime issues. I am using maven-shade 2.3.
I want to shade and relocate all dependencies so I do not specify any "includes" in the shade configuration. This causes the SimpleRelocater to relocate everything that does not specifically start with the excluded string.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>2.3</version>
<configuration>
<minimizeJar>false</minimizeJar>
<keepDependenciesWithProvidedScope>false</keepDependenciesWithProvidedScope>
<dependencyReducedPomLocation>${project.build.directory}/dependency-reduced-pom.xml
</dependencyReducedPomLocation>
<relocations>
<relocation>
<shadedPattern>shaded.</shadedPattern>
<excludes>
<exclude>my.project.package.**</exclude>
<exclude>java.**</exclude>
<exclude>javax.**</exclude>
<exclude>META-INF/**</exclude>
</excludes>
</relocation>
</relocations>
<artifactSet>
<excludes>
<exclude>javax.*</exclude>
<exclude>com.google.code.findbugs:jsr305</exclude>
</excludes>
</artifactSet>
<shadedArtifactAttached>false</shadedArtifactAttached>
</configuration>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
</execution>
</executions>
</plugin>
Here is a sample from my project:
public class UCDConstants
{
public static final Charset UTF8 = Charset.forName("utf-8");
}
After the maven-shade-plugin runs the charset is "shaded.utf-8." This causes runtime issues because that charset does not exist.
public class UCDConstants
{
public static final Charset UTF8 = Charset.forName("shaded.utf-8");
}
Elliott Neil Clark commented
Pretty sure that we're seeing this in https://issues.apache.org/jira/browse/HBASE-13889
We have the shad plugin configured like so:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<shadedArtifactAttached>false</shadedArtifactAttached>
<promoteTransitiveDependencies>true</promoteTransitiveDependencies>
<dependencyReducedPomLocation>${project.build.directory}/dependency-reduced-pom.xml</dependencyReducedPomLocation>
<artifactSet>
<includes>
<include>*:*</include>
</includes>
<excludes>
<exclude>org.apache.hbase:hbase-resource-bundle</exclude>
<exclude>org.slf4j:*</exclude>
<exclude>com.google.code.findbugs:*</exclude>
<exclude>com.github.stephenc.findbugs:*</exclude>
<exclude>org.apache.htrace:*</exclude>
<exclude>log4j:*</exclude>
<exclude>commons-logging:*</exclude>
</excludes>
</artifactSet>
<relocations>
<relocation>
<pattern>com</pattern>
<shadedPattern>org.apache.hadoop.hbase.shaded.com</shadedPattern>
<excludes>
<exclude>com.sun.**</exclude>
<exclude>com.ibm.**</exclude>
</excludes>
</relocation>
<relocation>
<pattern>net</pattern>
<shadedPattern>org.apache.hadoop.hbase.shaded.net</shadedPattern>
</relocation>
<relocation>
<pattern>io</pattern>
<shadedPattern>org.apache.hadoop.hbase.shaded.io</shadedPattern>
</relocation>
<relocation>
<pattern>org</pattern>
<shadedPattern>org.apache.hadoop.hbase.shaded.org</shadedPattern>
<excludes>
<exclude>org.apache.hadoop.**</exclude>
<exclude>org.apache.commons.logging.**</exclude>
<exclude>org.w3c.**</exclude>
<exclude>org.xml.**</exclude>
<exclude>org.slf4j.**</exclude>
</excludes>
</relocation>
</relocations>
<transformers>
<!-- Need to filter out some extraneous license files.
Don't use the ApacheLicenseRT because it just removes all
META-INF/LICENSE(.txt)? files, including ours. -->
<transformer implementation="org.apache.maven.plugins.shade.resource.DontIncludeResourceTransformer">
<resources>
<resource>LICENSE.txt</resource>
<resource>ASL2.0</resource>
</resources>
</transformer>
<!-- Where notices exist, just concat them -->
<transformer implementation="org.apache.maven.plugins.shade.resource.ApacheNoticeResourceTransformer">
<addHeader>false</addHeader>
<projectName>${project.name}</projectName>
</transformer>
</transformers>
</configuration>
</execution>
</executions>
</plugin>
This is relocating strings like: ``` "com.sun.security.auth.UnixPrincipal"
```"org/apache/hadoop/hbase/shaded/com.sun.security.auth.module.UnixLoginModule"
Kerem Yazici commented
we are having a similar issue where the below line
private static final String DEFAULT_TAG = "default";
replaced with
private static final String DEFAULT_TAG = "shaded.default";
Kerem Yazici commented
a bit more information,
so we have a tag that asks explicitly to shade the packages de as below
<relocation>
<pattern>de</pattern>
<shadedPattern>shaded.de</shadedPattern>
</relocation>
the actual package is de.undercouch, so changing the pattern from de to de.undercouch have fixed our issue.
Kerem Yazici commented
but we can't apply the same fix for the rx packages as it is the complete package name.
Dominik Süß commented
This one made me curse quite a lot the last hours - to work around I now "marked" the string and let the relocation undo this marker to restore the value I need. In my case this is about a tool that generates some configs while the class to be configured (Osgi Config) is being loaded in this tool as well. The name of the config contains the FQCN so it gets unintentionally relocated. I rarely have seen cases were the relocation of values within Strings is intended - there should be at least an optional parameter within the relocation rule to exclude occurrences in strings.
Eric Sirianni commented
I rarely have seen cases were the relocation of values within Strings is intended - there should be at least an optional parameter within the relocation rule to exclude occurrences in strings.
+1
Matteo Moci commented
Using this configuration https://issues.apache.org/jira/browse/MSHADE-156?focusedCommentId=15944705&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15944705
it seems this only happens when strings start with the relocation pattern:
public static final String A = "de-abc"; //this String is replaced with "shaded.de-abc"
public static final String B = "http://de-abc"; //this String is not changed
It happens with version 3.2.4
I'd argue that probably this is the intended behaviour to cover these use cases:
Class.forName("de.package.SomeClass");
If so, I would suggest to at least make this behaviour really explicit in the documentation, javadoc, etc...
Dan Ziemba commented
Probably related to the last line of SimpleRelocator::canRelocatePath. Seems like the path.startsWith( pathPattern ) needs to be a bit more complicated, like checking that is starts with the pattern immediately followed by period or slash?