Duplicated modules jar in stageMainDist

[iaco86]

Not sure this is an issue with my project structure or how the plugin stages jars, but wanted to report it here in order to get a little more visibility.

I haven't been able yet to create a small project to reproduce this, and I'm not a gradle expert, but our I noticed how the jar created for our project has duplicated libraries for all the project's modules.

Our project has the following simplified structure:

\ root
  |_ modules
    |_ core 

build.gradle

plugins {
    id 'org.gradle.playframework' version '0.10'
}
...
dependencies {
    implementation project(":core")
}

settings.gradle

include 'core'
project(':core').projectDir = new File(settingsDir, 'modules/core')

\modules\core\buld.gradle

plugins {
    id 'org.gradle.playframework'
}

Now, when I run the dist task, defined here, as a step to create our deployable RPM, that is creating a /build/stage/main/lib with the project library dependencies, and is copying twice the core's module jar and asset's jar, in a way that I have the following files:

1. core.jar
2. core-core.jar
3. core-assets.jar
4. core-core-assets.jar

where 1 and 2 contain the same exact files, same for 3 and 4.

I was able to pinpoint the task :stageMainDist as the task copying the dependencies there, but I'm not sure whether this behavior is an issue with the plugin or with our configuration.

Has anyone observed the same behavior?

Thanks!

[JLLeitschuh]

Are you able to create a minimal reproducer that demonstrates the issue?

[iaco86]

Hey @JLLeitschuh, I was able to pinpoint the issue to an incompatibility with the ospackage-application plugin: as soon as the plugin is added, the core.jar and core-assets.jar files are staged. Our project uses that plugin to create an RPM used to start a dockerized application.

Created this small project to verify that running the ./gradlew dist task will add the duplicated resources depending on the presence of the ospackage-application plugin.

Just out of curiosity, why is the renaming necessary for subprojects? Any suggestion on how to work around this?

Thanks!

[iaco86]

After some investigation, I'm going to answer my own question:

1. the problem is with the fact that both plugins add to the default main distribution of the project
2. the ospackage-application-plugin selects all the libraries and dependencies defined by the project, and so does the playframework plugin
3. on top of that, though, the playframework plugin through the org.gradle.playframework.plugins.PlayDistributionPlugin.PrefixArtifactFileNames class, renames dependencies from modules and libraries, essentially duplicating all the jars added to the distribution's lib

@JLLeitschuh would it be possible to make the "renaming" functionality configurable by the plugin? As of now my team is manually cleaning up the distribution libs before packaging the jar, but it's not ideal.

[JLLeitschuh]

@JLLeitschuh would it be possible to make the "renaming" functionality configurable by the plugin?

At this point, I'm more than willing to merge any PR that has sufficient unit & integration tests.

@big-guy do you know why this renaming occurs?

[big-guy]

This was something inherited from the old plugin. I think we started doing this because we had a user that had jars coming from subprojects with the same name as external dependencies (or maybe other subprojects). The identically named files would stomp on one another.

If we're adding both copies by default, that sounds like a bug.

[iaco86]

It's not a bug of the playframework plugin per se, but one that pops up if it is used with another plugin that modifies the project's standard distribution configuration.

I tested it locally and making the renaming configurable (default to true to avoid backward compatibility) solves the problem for us.

I'll try to write some tests for the new configuration and create a PR when I have some time.

Thanks!

[big-guy]

Sorry to take so long to get back to you. I see what's going on now.

nebula.ospackage-application also applies the built-in application plugin. This is ultimately what's causing problems and the duplicate jars are just one of the symptoms. I can reproduce the duplicate jar problem by just applying application.

What's happening is that both of these plugins apply the distribution plugin and have certain assumptions about how the distribution will be used. Both the play plugin and application plugin configure the "main" distribution and add similar content. They both create tasks to generate start scripts. This means there are deeper problems and duplicates. For example in bin/, you'll find two different copies of the start scripts. In your example, the "sandbox" project actually has three different jars (assets, production code, runnable jar). All three are getting copied, but the production code one overwrites the runnable jar, so one set of the start scripts won't even work. I suspect the order that this overwrite happens depends on the order the plugins are applied.

Since the issue is more than just the renaming behavior, I think the proper fix is more involved. I think the fix is to change PlayDistributionPlugin to apply application and remove as much of the specialness as possible. The reason this wasn't done originally is that the original Play plugin copied a lot of the functionality from existing plugins (distribution, application, etc) instead of applying those plugins directly. I think it's still possible to do the PrefixArtifactFileNames trick and runnable jar as part of the "main" distribution by replacing/reconfiguring what the application plugin adds.

In the mean time, you may be able to workaround this by just applying nebula.ospackage and configuring it to package up the Play application's distribution. This would avoid the application plugin from being applied.

[iaco86]

Hey @big-guy thanks for explaining this - I'll see if I have more time to dig into this, but as of now we're working around the mis-configured distribution to remove files based on regexes, and that help us cutting the size of our image in half.

Unfortunately we use both plugins (playframework to start local versions of the service using gradle and nebula.ospackage-application to package a docker image to push for production use), so I'm not sure we can simply avoid applying both plugins.

I'll try to understand you suggestion a bit better - I'm still pretty new to the whole gradle world - and see if we can achieve something by means of configuration only.

Thanks for the feedback!

[Jonatha1983]

Hi @iaco86 and @big-guy

I think I have a very similar issue, just wondered if you know why when running runPlay the app is working but when running from the installation script it fail ? what is the main different there ?

Thanks