maven-shade-plugin
maven-shade-plugin copied to clipboard
apache
[MSHADE-206] Multi-module builds don't pick up dependency-reduced-pom.xml
Robert Joseph Evans opened MSHADE-206 and commented
I'm not sure if there is a good solution for this or not. As part of the Apache Storm project we are trying to shade a lot of dependencies for the core platform (storm-core), but we also have other modules in the same build that depend on storm-core, but use the shade plugin to create an uber jar for deployment. The problem is that if we build all of the modules at once and let maven handle resolving dependencies the uber jar projects don't see storm-core's dependency-reduced-pom.xml. They see the original pom.xml and end up not packaging dependencies that they need.
Our "fix" for this https://github.com/apache/storm/pull/736 is to split the build up into two phases. One that just builds storm-core and another that builds the rest. This is going to be a real pain for developers. Are we doing something wrong? Is this working as expected?
I am happy to submit a fix for this, but I am far from an expert on maven so if someone in this project sees a direction that I can go in to fix this I would appreciate any guidance you might have.
Issue Links:
-
FLINK-3158 Shading does not remove google guava from flink-dist fat jar
-
MSHADE-326 Hide shaded dependencies from the rest of the reactor build
-
MNG-5899 Reactor is no longer using the dependency reduced pom created by maven-shade-plugin ("depends upon")
10 votes, 13 watchers
Trask Stalnaker commented
I'm experiencing possibly the same issue. For me, it actually works fine if I build with maven 2.2.5, but does not work if I build with maven 3.3.1 and 3.3.3. Can you see if this is the case for you also?
Robert Joseph Evans commented
I'll see if I can test that out. Our build explicitly relies on maven 3.x, so I'll have to find out what it is doing that make the older maven not work.
Trask Stalnaker commented
Oh sorry, that was a typo on my part, I meant that it works for me with 3.2.5, but fails with 3.3.1 and 3.3.3.
Trask Stalnaker commented
I did some bisecting and found the maven commit that broke this for me:
https://github.com/apache/maven/commit/be3fb200326208ca4b8c41ebf16d5ae6b8049792
In particular, that commit introduced some code to not rebuild the model if it was already loaded during reactor resolution.
I think that is causing maven not to use the updated "dependency reduced" pom that has been created by the maven-shade-plugin.
Removing this code fixes the situation for me. I'm hoping that code was just introduced as a performance optimization, and that I haven't broken anything functionally by removing it (the tests still pass at least).
I forked the maven project and pushed the fix here if you want to try it: https://github.com/trask/maven/tree/MSHADE-206
I think we will need to submit this issue to the maven project, but I will wait a bit for your analysis to see if you are facing the same issue.
Thanks, Trask
Robert Joseph Evans commented
Thanks is great news. I will try and build/test with your updated version of maven. I'll let you know ASAP if it fixes the issue.
Robert Joseph Evans commented
Yes you are correct that did fix the issue and put it all back to the way it should be. Thanks for jumping on this so quickly. I will just make sure that we use 3.2.5 or below for our builds, or once your fix goes in we can switch to the newer version.
Francois MAROT commented
Hello Elliotte Rusty Harold , as a simple Maven user but following closely what happens in Maven world and using the shade plugin, I can tell you this is still an issue and not resolved. If you use the shade plugin, the module using the shade plugin has to be the LAST one in your Maven execution. Meaning that no module in your multi-module should depend on the shade jar or otherwise you will not get what you expect to get regarding dependencies. You have to split your build in two phase where the first installs (or deploys) the shaded jar.