Create separate shadowJars for code and dependencies

[jayadev]

Using the awesome shadow plugin version 4.0.2 and gradle 4.10, I wanted to create two separate shadowJars, one for my source code and another for my dependencies (since dependencies are large and rarely changes I don’t want to repackage them every time I change my source code). What I have in mind is to have a gradle plugin that add two separate tasks and which takes the same configurations supplied by user for shadowJar and overrides the configurations/sources used to create the shadowJar.

Below is what I have got so far, still trying to figure out a clean way to pass the shadow configs only once and whether there are other gotchas I need to worry about (ex: having two mergeServiceFiles will break etc)

import com.github.jengelman.gradle.plugins.shadow.tasks.ShadowJar

task dependencyShadowJar(type: ShadowJar) {
    mergeServiceFiles()
    zip64 = true
    relocate 'com.google.common', 'com.shadow.google.common'
    classifier = 'dependencies'
    configurations = [project.configurations.runtime]
}

task userCodeShadowJar(type: ShadowJar) {
    mergeServiceFiles()
    zip64 = true
    relocate 'com.google.common', 'com.shadow.google.common'
    classifier = 'mycode'
    from sourceSets.main.output
}

task splitShadowJar {
    doLast {
        println "Building separate src and dependency shadowJars"
    }
}

splitShadowJar.dependsOn dependencyShadowJar
dependencyShadowJar.dependsOn userCodeShadowJar

• Ideally I would like to have a shadowJar settings specified once and the tasks copies the same settings, does that require creating a custom Plugin task in groovy ?
• Can I copy the settings from existing shadowJar that user specifies and just overrides the from or configurations part alone for my purpose

Anybody has attempted something similar ?

[johnrengelman]

That's a really interesting way to do this. Do the relocations apply correctly to the source files?

[jayadev]

@johnrengelman doing a quick inspection of the strings in the class file, it looks like the relocation is applied in the source files. So it seems to do the right thing. I wanted to make this reusable across many projects so package it as a gradle plugin for easy usage.

Ideally I would like :

• the users to specify their normal shadowJar configs in build.gradle.
• applying my plugin would add these new tasks that take the same shadowJar configs and use it to create the two jars which can be used as needed.

I was hoping I could achieve this by extending what I have started with and some way copying the settings from shadowJar and calling the ShadowJar task with my updated settings. Is that doable using the above approach or will I have to create my own gradle task in groovy, any pointers ?

[dpkirchner]

I think I've got a solution -- at least, it's a solution that works for me. I've adapted jayadev's example to fit my needs:

import com.github.jengelman.gradle.plugins.shadow.tasks.ShadowJar

// This just places the dependencies in a jar, not the app files. This is
// a partial copy of what the default shadowJar task does.
//
// This produces a jar named ${baseName}-dependencies-${version}.jar
task dependencyShadowJar(type: ShadowJar) {
    mergeServiceFiles()

    zip64 true

    relocate 'com.google.common', 'shadow.com.google.common'
    relocate 'com.google.protobuf', 'shadow.com.google.protobuf'

    destinationDirectory = file("build/libs")
    setAppendix("dependencies")

    configurations = [project.configurations.runtimeClasspath]
    exclude('META-INF/INDEX.LIST', 'META-INF/*.SF', 'META-INF/*.DSA', 'META-INF/*.RSA', 'module-info.class')
}

// This includes only files in the runtime directories (I think?), so just
// your app's code.
//
// This produces a jar named ${baseName}-app-${version}.jar
task appShadowJar(type: ShadowJar) {
    mergeServiceFiles()

    manifest {
        attributes 'Implementation-Title': 'example',
                'Implementation-Version': '0.0.1',
                'Main-Class': 'com.example.app'
    }

    zip64 true

    relocate 'com.google.common', 'shadow.com.google.common'
    relocate 'com.google.protobuf', 'shadow.com.google.protobuf'

    destinationDirectory = file("build/libs")
    setAppendix("app")

    from sourceSets.main.output
    configurations = [project.configurations.runtime]
}

tasks.assemble.dependsOn dependencyShadowJar
tasks.assemble.dependsOn appShadowJar

// We don't need the jar task any more.
task jar(type: Jar, overwrite: true) {
    // no-op
}

I'm not a gradle pro, so there's a darn good chance I'm not covering all scenarios I should.

[jschneider]

I run into the same problem. Thanks for your work - it helped me solving it.

The importat line is this: configurations = [project.configurations.runtimeClasspath]

Would love to see this supported out of the box in shadowJars

[darylsze]

@jschneider add into main / module's build.gradle

configurations {
    runtimeClasspath // just define a separate configuration
}

[ctadlock]

I think I found an easier way to do this, at least in the case where you can filter by the ResolvedDependancy.

In my case I dont want two shadowed jars, just one for "my" app projects; the rest are fine as individual jars like usual without shadow.

Example to only include dependencies that are from Acme:

tasks.withType<ShadowJar> {
    dependencies {
        this.exclude {
            val exclude = it.name.contains("Acme", true).not()

            // helpful for debugging
            if (exclude) {
                println("Excluding: ${it.name}")
            } else {
                println("\tIncluding: ${it.name}")
            }

            exclude
        }
    }
}