[MSHADE-251] Add skip parameter to shade goal

[jira-importer]

Jonathan Haber opened MSHADE-251 and commented

Most Maven plugins have a skip parameter but not the shade plugin. Our use-case is that we have multiple ways of packaging apps for deployment (using the shade plugin is one such way). We have these packaging plugins defined in our shared parent POM and projects can use Maven properties to control which ones to skip and which to execute.

We've added this feature on our fork (which you can view here). It's not much code and having a skip parameter can be a lifesaver when you need it, so I was hoping this feature could get added. Let me know what you think.

Thanks, Jonathan

---

2 votes, 5 watchers

[jira-importer]

Björn Michael commented

Examples with skip parameter

• spring-boot:repackage
• assembly:single
• jar:test-jar
• war:war

[jira-importer]

Robert Scholte commented

I am aware that there are a lot of plugins with the skip-parameter and that the code is quite simple to add, but it is in general a codesmell, especially in packaging related plugins. The effect is that Maven will initialize the plugin, make it part of the buildplan, while line 1 would say: "skip me". That's to me a waste of resources. So I'll vote -1 and hope we can find a better way to solve this. Have you tried adding the execution-block in pluginManagement, so you only have to add the plugins groupId+artifactId to the targeted modules?

[jira-importer]

Jonathan Haber commented

We have over 6,000 modules within our codebase, roughly 2,400 of which are packaged for deployment. Embedding this sort of logic directly in each of these modules creates a maintenance headache as we update things over time. Instead, we centralize all of this in our shared parent POM which makes maintenance much more sane and ensures that all of our modules are using the same standardized build lifecycle.

In the case of packaging, applications can put a file at the module root called ".build-executable" which indicates that the module should be packaged for deployment and activates a Maven profile (we inherited this concept from the brilliant [basepom project|https://github.com/basepom/basepom/blob/ab2a40c85aa32c37069a9a03fb4594078db08bcc/foundation/pom.xml#L1043-L1093]).

If we always used the shade plugin to package things for deployment, then this would be fine. We would just define the shade plugin inside this executable profile, and all would be well. However, we moved on from that approach in 2016 because of the problems I outlined here:

https://product.hubspot.com/blog/the-fault-in-our-jars-why-we-stopped-building-fat-jars

So we actually have at least 3 distinct ways to package things for deployment (only one of which involves the shade plugin). Which strategy we use depends on the environment the build is running in (CI vs. local, for example), and can also be controlled by Maven properties (which can't currently be used to activate/deactivate Maven profiles).

Which is a longwinded way of trying to express that there are companies using Maven at very large scales with complex workflows and adding a skip parameter to every plugin is a very low-cost way to help keep things flexible for these advanced use-cases.

[jira-importer]

Robert Scholte commented

Can you attach a small multimodule of poms and .build-executable to show how this works or should work. Sometimes code says more than text.

[jira-importer]

Björn Michael commented

The best known example originates from Maven Surefire Plugin providing skipTests User property. IMO a new skip parameter would increase flexibilty a lot and would outweigh codesmell concerns. E.g. it would allow to skip the execution of an existing shade plugin definition within a project without touching pom.xml at command line.

My usecase - usage of skip parameter in core / common module

1. Parent project contains a build-fat profile containing shade plugin definition.
2. Reactor comprises of 12 modules each has one Main class and 1 core / common module without a Main class i.e. should not use shade plugin.
3. Build whole reactor with -Pbuild-fat but exclude this profile for core module A simple skip parameter in core / common module would be completely sufficient.

[jira-importer]

Jonathan Haber commented

I pushed an example project to GitHub here:

https://github.com/jhaber/MSHADE-251

If you want to play with it, you can clone it and run "mvn clean verify" from the root directory.

At the top level there are two folders. The first folder is called packaging-plugin; this folder contains a Maven plugin which would be owned by our infrastructure team.

The second folder is called example-project. For simplicity I just embedded the plugin configuration directly in its POM, but in reality all of this plugin configuration would be centralized in our parent POM that our applications extend: https://github.com/jhaber/MSHADE-251/blob/c534a3a31fef9dd4d9369c1bd1f1281ed5baee29/example-project/pom.xml#L23-L147

There you can see the executable profile activated by a .build-executable file:

https://github.com/jhaber/MSHADE-251/blob/c534a3a31fef9dd4d9369c1bd1f1281ed5baee29/example-project/pom.xml#L47-L52

This profile triggers the packaging-plugin to run, it also modifies some configuration on the JAR plugin, and finally it runs 3 separate packaging plugins.

There is the shade plugin which builds a fat JAR, the slimfast plugin upload goal (uploads dependencies to S3), and the slimfast plugin copy goal (copies dependencies to target folder). We only want to actually execute one of these 3 goals. The job of the packaging-plugin is to inspect the currently building module, determine which packaging mode is appropriate, and set the corresponding Maven properties so that the desired plugin executes and the others get skipped. For example, it sets these properties if packaging as a fat JAR is desired:

https://github.com/jhaber/MSHADE-251/blob/c534a3a31fef9dd4d9369c1bd1f1281ed5baee29/packaging-plugin/src/main/java/com/packaging/ConfigurePackagingMojo.java#L44-L48

It determines which packaging mode to use based on the type of the deployable being built, which is set as a Maven property, as well as the environment that the build is happening in (for example, we only want to upload dependencies to S3 when building inside our CI environment). Similarly, maybe we package Spark jobs and Lambda functions as fat JARs for convenience:

https://github.com/jhaber/MSHADE-251/blob/c534a3a31fef9dd4d9369c1bd1f1281ed5baee29/packaging-plugin/src/main/java/com/packaging/ConfigurePackagingMojo.java#L14-L16

But other deployable types can use a more efficient packaging mode. These deployable types are set as properties in the POM, which can't be used to influence profile activation:

https://github.com/jhaber/MSHADE-251/blob/c534a3a31fef9dd4d9369c1bd1f1281ed5baee29/example-project/example-rest-api/pom.xml#L15

And crucially, as a user of this system you don't have to worry about any of these details. You just say that you're building a spark job and the infrastructure handles packaging that appropriately. For example, maybe the infrastructure team implements some optimization so that we no longer need to build fat JARs for Spark. With this setup, no user code needs to get updated. We would only need to update the logic in the packaging plugin, which is also owned by the infrastructure team.

In reality this is all at a much larger scale and more complex, but hopefully this example is helpful

[jira-importer]

Björn Michael commented

A skip parameter exists in several plugins, is a common idiom in maven ecosystem and increases flexibility a lot. Furthermore changes to the code are little. Please rethink your stance.

[jira-importer]

Andreas Rauchberger commented

i do see the <skip> parameter more or less as a default maven-plugin feature/&strategy.

it really makes things easier. we do use the parent-pom-approach a lot to generalise things.

we then use profiles (mainly setting properties there) to do things like "build-only", "build-and-docker", "build-release", ... controlling which plugins to execute.

i think this is a common approach, and prevents the user from knowing/providing all properties on command line.

[jira-importer]

Delany commented

Robert Scholte again you raise the issue of code smells, but this is a usability issue. I want to add the plugin, get it into source and test it out, and when its ready I'll flip the switch. The real code smell is that Maven doesn't provide a skip config for all plugins by default

[jira-importer]

Robert Scholte commented

Any parameter that can be set from commandline AND effects the result of the artifact should be banned. Otherwise we'll get the other discussions: why doesn't my project compile( or even worse  .. run) anymore? Oh, because somebody added -Dshade.skip and now the classes aren't relocated.

As one of the plugin maintainer I'm not giong to take responsibility for that at a plugin level. You are allowed to do it on a project level by adding profiles to the pom. Having reliable builds is much more important compared to flexibility with skip parameters.

Regarding inheritence (specify the plugin in a parent pom, and only let a subset use it) is already possible when pluginManagement, but it could be improved.

[jira-importer]

Delany commented

I don't know what world you're living in where people randomly add parameters to command lines, my colleagues all have keyboard-phobia, they'd do anything to use a shorter build command.

Imagine if we were making desk lamps without on/off switches. People must pull it out of the wall? Ridiculous.

And as for profiles, thats exactly what I've done. Hundreds of profiles for switching plugins and off. But of course I can't just turn on a profile by default, because that would turn off the other profiles that were turned on by default, hence they all have

<activation>   <property>     <name>!muaeixubuayol</name>   </property> </activation>

And that's when I didn't just bind the plugin to a non-existent phase! Which makes three ways that plugins are being toggled.

[jira-importer]

Andreas Rauchberger commented

I'm using profiles a lot, for controlling wether to execute a plugin in certain circumstances or not.

e.g. sign the code for production but skip it for dev, ... and so on. or skip shade/docker if we need to deliver a Main+shell-script (or a windows service)

the basic approach i'm following is to have a build-pipeline in maven containing anything needed, then use profiles to disable certain plugin executions if there is no current need.

what i do see as a massive overhead is declaring the same build steps in each profile to just being able to skip the one currently not needed - this is a guarantee for inconsistence the later refactoring will happen.

i east don't get the problem adding a skip-switch like 95% of maven-plugins already do.

[jira-importer]

Andreas Rauchberger commented

has this been solved in MSHADE-382Add an option to skip execution?

why i can't pull a more recent version of the plugin then 3.2.4?