gradle-dependency-export
gradle-dependency-export copied to clipboard
uklance
Not including gradle plugin dependencies
My project has plugin dependencies declared using the newer plugins DSL but these do not appear to be included in the set of dependencies. Is there any way to include them? I can enumerate them correctly using the built-in buildEnvironment task, but do not know if there's a way to include them via your plugin.
I assumed these would are added to buildscript.configurations.classpath. Can you try declaring using the old plugin style?
Perhaps there's another ConfigurationContainer somewhere that I'm not adding by default?
I wonder if the plugin dependencies are added to the buildSrc project when using the new plugin style declaration?
They DO appear to be enumerating through buildscript.configurations.classpath. However, bizarrely, your plugin appears to selectively grab the plugins. For example, my DSL declaration has this:
plugins {
id 'com.palantir.git-version' version '0.12.0-rc2'
id 'com.github.johnrengelman.shadow' version "4.0.4"
id 'java'
id 'distribution'
id "com.lazan.dependency-export" version "0.5"
}
Out of that set, the only one that makes it into the export is shadow-4.0.4. Neither git-version nor your own plugin were exported. The only commonality I see is perhaps the version numbers start with "0.". Could that be somehow causing the issue?
Can you please take your zip task out of the equation and just try invoking my task which copies to the folder?
I see the problem now. It put the plugins under an extra gradle/plugins/ parent folder in the export. They are no placed into the same folder structure as the rest of the maven repo. So this ends up not lining up with the package name and, hence, improper maven structure. Unless gradle plugins are supposed to somehow go into that extra "gradle/plugin" folder for maven structure. It also did not export anything but the JAR file for plugins, even though I had sources and javadoc enabled.
This is somewhat bizarre since shadow did end up in the right place. But gradle-dependency-export and gradle-git-version ended up in a `/gradle/plugins/" subdirectory in the export.
For the record, I stopped testing with my zip task a while ago. I am testing purely with exports to the file system.
If this helps, here is the full output from buildEnvironment task:
classpath
+--- com.palantir.git-version:com.palantir.git-version.gradle.plugin:0.12.0-rc2
| \--- gradle.plugin.com.palantir.gradle.gitversion:gradle-git-version:0.12.0-rc2
| +--- org.eclipse.jgit:org.eclipse.jgit:4.5.4.201711221230-r
| | +--- com.jcraft:jsch:0.1.53
| | +--- com.googlecode.javaewah:JavaEWAH:0.7.9
| | +--- org.apache.httpcomponents:httpclient:4.3.6
| | | +--- org.apache.httpcomponents:httpcore:4.3.3
| | | +--- commons-logging:commons-logging:1.1.3
| | | \--- commons-codec:commons-codec:1.6
| | \--- org.slf4j:slf4j-api:1.7.2
| \--- com.google.guava:guava:20.0
+--- com.github.johnrengelman.shadow:com.github.johnrengelman.shadow.gradle.plugin:4.0.4
| \--- com.github.jengelman.gradle.plugins:shadow:4.0.4
| +--- org.jdom:jdom2:2.0.6
| +--- org.ow2.asm:asm:7.0-beta
| +--- org.ow2.asm:asm-commons:7.0-beta
| | +--- org.ow2.asm:asm:7.0-beta
| | +--- org.ow2.asm:asm-tree:7.0-beta
| | | \--- org.ow2.asm:asm:7.0-beta
| | \--- org.ow2.asm:asm-analysis:7.0-beta
| | \--- org.ow2.asm:asm-tree:7.0-beta (*)
| +--- commons-io:commons-io:2.5 -> 2.6
| +--- org.apache.ant:ant:1.9.7
| | \--- org.apache.ant:ant-launcher:1.9.7
| +--- org.codehaus.plexus:plexus-utils:3.0.24 -> 3.1.0
| +--- org.apache.logging.log4j:log4j-core:2.11.0
| | \--- org.apache.logging.log4j:log4j-api:2.11.0
| \--- org.vafer:jdependency:2.1.1
| \--- commons-io:commons-io:2.6
\--- com.lazan.dependency-export:com.lazan.dependency-export.gradle.plugin:0.5
\--- gradle.plugin.com.lazan:gradle-dependency-export:0.5
+--- org.apache.maven:maven-model:3.5.2
| +--- org.codehaus.plexus:plexus-utils:3.1.0
| \--- org.apache.commons:commons-lang3:3.5
\--- org.apache.maven:maven-model-builder:3.5.2
+--- org.codehaus.plexus:plexus-utils:3.1.0
+--- org.codehaus.plexus:plexus-interpolation:1.24
+--- org.codehaus.plexus:plexus-component-annotations:1.7.1
+--- org.apache.maven:maven-model:3.5.2 (*)
+--- org.apache.maven:maven-artifact:3.5.2
| +--- org.codehaus.plexus:plexus-utils:3.1.0
| \--- org.apache.commons:commons-lang3:3.5
+--- org.apache.maven:maven-builder-support:3.5.2
| \--- org.apache.commons:commons-lang3:3.5
+--- com.google.guava:guava:20.0
\--- org.apache.commons:commons-lang3:3.5
and here is the resulting maven-dependency-export folder:
+---com
| +---github
| | \---jengelman
| | \---gradle
| | \---plugins
| | \---shadow
| | \---4.0.4
| | shadow-4.0.4-javadoc.jar
| | shadow-4.0.4-sources.jar
| | shadow-4.0.4.jar
| | shadow-4.0.4.pom
| |
| +---google
| | \---guava
| | +---guava
| | | \---20.0
| | | guava-20.0-javadoc.jar
| | | guava-20.0-sources.jar
| | | guava-20.0.jar
| | | guava-20.0.pom
| | |
| | \---guava-parent
| | \---20.0
| | guava-parent-20.0.pom
| |
| +---googlecode
| | \---javaewah
| | \---JavaEWAH
| | \---0.7.9
| | JavaEWAH-0.7.9-javadoc.jar
| | JavaEWAH-0.7.9-sources.jar
| | JavaEWAH-0.7.9.jar
| | JavaEWAH-0.7.9.pom
| |
| \---jcraft
| \---jsch
| \---0.1.53
| jsch-0.1.53-javadoc.jar
| jsch-0.1.53-sources.jar
| jsch-0.1.53.jar
| jsch-0.1.53.pom
|
+---commons-codec
| \---commons-codec
| \---1.6
| commons-codec-1.6-javadoc.jar
| commons-codec-1.6-sources.jar
| commons-codec-1.6.jar
| commons-codec-1.6.pom
|
+---commons-io
| \---commons-io
| \---2.6
| commons-io-2.6-javadoc.jar
| commons-io-2.6-sources.jar
| commons-io-2.6.jar
| commons-io-2.6.pom
|
+---commons-logging
| \---commons-logging
| \---1.1.3
| commons-logging-1.1.3-javadoc.jar
| commons-logging-1.1.3-sources.jar
| commons-logging-1.1.3.jar
| commons-logging-1.1.3.pom
|
+---gradle
| \---plugin
| \---com
| +---lazan
| | \---gradle-dependency-export
| | \---0.5
| | gradle-dependency-export-0.5.jar
| |
| \---palantir
| \---gradle
| \---gitversion
| \---gradle-git-version
| \---0.12.0-rc2
| gradle-git-version-0.12.0-rc2.jar
|
\---org
+---apache
| +---ant
| | +---ant
| | | \---1.9.7
| | | ant-1.9.7-javadoc.jar
| | | ant-1.9.7-sources.jar
| | | ant-1.9.7.jar
| | | ant-1.9.7.pom
| | |
| | +---ant-launcher
| | | \---1.9.7
| | | ant-launcher-1.9.7-sources.jar
| | | ant-launcher-1.9.7.jar
| | | ant-launcher-1.9.7.pom
| | |
| | \---ant-parent
| | \---1.9.7
| | ant-parent-1.9.7.pom
| |
| +---apache
| | +---13
| | | apache-13.pom
| | |
| | +---17
| | | apache-17.pom
| | |
| | +---18
| | | apache-18.pom
| | |
| | \---9
| | apache-9.pom
| |
| +---commons
| | +---commons-lang3
| | | \---3.5
| | | commons-lang3-3.5-javadoc.jar
| | | commons-lang3-3.5-sources.jar
| | | commons-lang3-3.5.jar
| | | commons-lang3-3.5.pom
| | |
| | \---commons-parent
| | +---22
| | | commons-parent-22.pom
| | |
| | +---28
| | | commons-parent-28.pom
| | |
| | +---41
| | | commons-parent-41.pom
| | |
| | \---42
| | commons-parent-42.pom
| |
| +---httpcomponents
| | +---httpclient
| | | \---4.3.6
| | | httpclient-4.3.6-javadoc.jar
| | | httpclient-4.3.6-sources.jar
| | | httpclient-4.3.6.jar
| | | httpclient-4.3.6.pom
| | |
| | +---httpcomponents-client
| | | \---4.3.6
| | | httpcomponents-client-4.3.6.pom
| | |
| | +---httpcomponents-core
| | | \---4.3.3
| | | httpcomponents-core-4.3.3.pom
| | |
| | +---httpcore
| | | \---4.3.3
| | | httpcore-4.3.3-javadoc.jar
| | | httpcore-4.3.3-sources.jar
| | | httpcore-4.3.3.jar
| | | httpcore-4.3.3.pom
| | |
| | \---project
| | \---7
| | project-7.pom
| |
| +---logging
| | +---log4j
| | | +---log4j
| | | | \---2.11.0
| | | | log4j-2.11.0.pom
| | | |
| | | +---log4j-api
| | | | \---2.11.0
| | | | log4j-api-2.11.0-javadoc.jar
| | | | log4j-api-2.11.0-sources.jar
| | | | log4j-api-2.11.0.jar
| | | | log4j-api-2.11.0.pom
| | | |
| | | \---log4j-core
| | | \---2.11.0
| | | log4j-core-2.11.0-javadoc.jar
| | | log4j-core-2.11.0-sources.jar
| | | log4j-core-2.11.0.jar
| | | log4j-core-2.11.0.pom
| | |
| | \---logging-parent
| | \---1
| | logging-parent-1.pom
| |
| \---maven
| +---maven
| | \---3.5.2
| | maven-3.5.2.pom
| |
| +---maven-artifact
| | \---3.5.2
| | maven-artifact-3.5.2-javadoc.jar
| | maven-artifact-3.5.2-sources.jar
| | maven-artifact-3.5.2.jar
| | maven-artifact-3.5.2.pom
| |
| +---maven-builder-support
| | \---3.5.2
| | maven-builder-support-3.5.2-javadoc.jar
| | maven-builder-support-3.5.2-sources.jar
| | maven-builder-support-3.5.2.jar
| | maven-builder-support-3.5.2.pom
| |
| +---maven-model
| | \---3.5.2
| | maven-model-3.5.2-javadoc.jar
| | maven-model-3.5.2-sources.jar
| | maven-model-3.5.2.jar
| | maven-model-3.5.2.pom
| |
| +---maven-model-builder
| | \---3.5.2
| | maven-model-builder-3.5.2-javadoc.jar
| | maven-model-builder-3.5.2-sources.jar
| | maven-model-builder-3.5.2.jar
| | maven-model-builder-3.5.2.pom
| |
| \---maven-parent
| \---27
| maven-parent-27.pom
|
+---codehaus
| \---plexus
| +---plexus
| | \---4.0
| | plexus-4.0.pom
| |
| +---plexus-component-annotations
| | \---1.7.1
| | plexus-component-annotations-1.7.1-javadoc.jar
| | plexus-component-annotations-1.7.1-sources.jar
| | plexus-component-annotations-1.7.1.jar
| | plexus-component-annotations-1.7.1.pom
| |
| +---plexus-containers
| | \---1.7.1
| | plexus-containers-1.7.1.pom
| |
| +---plexus-interpolation
| | \---1.24
| | plexus-interpolation-1.24-javadoc.jar
| | plexus-interpolation-1.24-sources.jar
| | plexus-interpolation-1.24.jar
| | plexus-interpolation-1.24.pom
| |
| \---plexus-utils
| \---3.1.0
| plexus-utils-3.1.0-javadoc.jar
| plexus-utils-3.1.0-sources.jar
| plexus-utils-3.1.0.jar
| plexus-utils-3.1.0.pom
|
+---eclipse
| \---jgit
| +---org.eclipse.jgit
| | \---4.5.4.201711221230-r
| | org.eclipse.jgit-4.5.4.201711221230-r-javadoc.jar
| | org.eclipse.jgit-4.5.4.201711221230-r-sources.jar
| | org.eclipse.jgit-4.5.4.201711221230-r.jar
| | org.eclipse.jgit-4.5.4.201711221230-r.pom
| |
| \---org.eclipse.jgit-parent
| \---4.5.4.201711221230-r
| org.eclipse.jgit-parent-4.5.4.201711221230-r.pom
|
+---jdom
| \---jdom2
| \---2.0.6
| jdom2-2.0.6-javadoc.jar
| jdom2-2.0.6-sources.jar
| jdom2-2.0.6.jar
| jdom2-2.0.6.pom
|
+---ow2
| +---asm
| | +---asm
| | | \---7.0-beta
| | | asm-7.0-beta-javadoc.jar
| | | asm-7.0-beta-sources.jar
| | | asm-7.0-beta.jar
| | | asm-7.0-beta.pom
| | |
| | +---asm-analysis
| | | \---7.0-beta
| | | asm-analysis-7.0-beta-javadoc.jar
| | | asm-analysis-7.0-beta-sources.jar
| | | asm-analysis-7.0-beta.jar
| | | asm-analysis-7.0-beta.pom
| | |
| | +---asm-commons
| | | \---7.0-beta
| | | asm-commons-7.0-beta-javadoc.jar
| | | asm-commons-7.0-beta-sources.jar
| | | asm-commons-7.0-beta.jar
| | | asm-commons-7.0-beta.pom
| | |
| | \---asm-tree
| | \---7.0-beta
| | asm-tree-7.0-beta-javadoc.jar
| | asm-tree-7.0-beta-sources.jar
| | asm-tree-7.0-beta.jar
| | asm-tree-7.0-beta.pom
| |
| \---ow2
| \---1.5
| ow2-1.5.pom
|
+---slf4j
| +---slf4j-api
| | \---1.7.2
| | slf4j-api-1.7.2-javadoc.jar
| | slf4j-api-1.7.2-sources.jar
| | slf4j-api-1.7.2.jar
| | slf4j-api-1.7.2.pom
| |
| \---slf4j-parent
| \---1.7.2
| slf4j-parent-1.7.2.pom
|
+---sonatype
| +---forge
| | \---forge-parent
| | \---10
| | forge-parent-10.pom
| |
| \---oss
| \---oss-parent
| +---5
| | oss-parent-5.pom
| |
| +---6
| | oss-parent-6.pom
| |
| \---7
| oss-parent-7.pom
|
\---vafer
\---jdependency
\---2.1.1
jdependency-2.1.1-javadoc.jar
jdependency-2.1.1-sources.jar
jdependency-2.1.1.jar
jdependency-2.1.1.pom
From what I can tell, the problem is possibly that the export is missing some of the pom-only dependencies, such as: https://plugins.gradle.org/m2/com/palantir/git-version/com.palantir.git-version.gradle.plugin/0.12.0-rc2/com.palantir.git-version.gradle.plugin-0.12.0-rc2.pom
I've found some weirdness with the new plugin declaration style. Eg I'm only able to use a plugin as an included build using the old style plugin declaration. Have you tried declaring using the old style?
I'm not sure about the gradle.plugin artifacts, perhaps the plugin publish libraries created this for my plugin (and others)
the gradle.plugin artifacts are the real artifacts for this plugin. They are tied together by the parent POM:
+--- com.palantir.git-version:com.palantir.git-version.gradle.plugin:0.12.0-rc2
| \--- gradle.plugin.com.palantir.gradle.gitversion:gradle-git-version:0.12.0-rc2
| +--- org.eclipse.jgit:org.eclipse.jgit:4.5.4.201711221230-r
| | +--- com.jcraft:jsch:0.1.53
| | +--- com.googlecode.javaewah:JavaEWAH:0.7.9
| | +--- org.apache.httpcomponents:httpclient:4.3.6
| | | +--- org.apache.httpcomponents:httpcore:4.3.3
| | | +--- commons-logging:commons-logging:1.1.3
| | | \--- commons-codec:commons-codec:1.6
| | \--- org.slf4j:slf4j-api:1.7.2
The real JAR file is actually under gradle.plugin.com.palantir.gradle.gitversion as listed above and that is how it actually appears on the maven server:
https://plugins.gradle.org/m2/gradle/plugin/com/palantir/gradle/gitversion/gradle-git-version/0.12.0-rc2/
For whatever reason, your plugin appears to have properly exported the JAR file, but not the parent POM as the dependencies are declared. Not sure why the POM-only top level was disregarded, since it does appear to export some POM-only dependencies such as oss-parent.
There was a reason I had to move over to the new style versus the old build-script style, but I cannot remember why now. I will see if that makes a difference.
I'm now thoroughly confused. I switched over to the old-style buildscript declaration and now I"m getting failures attempting to pull your plugin as a dependency. Apparently, this URL no longer exists:
https://plugins.gradle.org/m2/com/lazan/gradle-dependency-export/0.5/
Only version 0.2 appears to be published to https://plugins.gradle.org/m2/com/lazan/gradle-dependency-export/.
Ahh, looks this one exists instead: https://plugins.gradle.org/m2/com/lazan/dependency-export/com.lazan.dependency-export.gradle.plugin/0.5/ Your readme instructions have a slight typo in the old-style buildscript declaration. This:
dependencies {
classpath "com.lazan:gradle-dependency-export:0.5"
}
needs to be this:
dependencies {
classpath "com.lazan:dependency-export:0.5"
}
I'm totally lost with the differences in how these plugins are deployed and represented on the servers, depending upon whether the syntax is the old or new style. In the old style, your plugin is actually at:
dependencies {
classpath "gradle.plugin.com.lazan:gradle-dependency-export:0.5"
}
and not:
dependencies {
classpath "com.lazan:gradle-dependency-export:0.5"
}
Yet, the "new" style seemed to work with:
plugins {
id "com.lazan.dependency-export" version "0.5"
}
So which one is correct? Is the structure at gradle.plugin.com.lazan or just com.lazan?
Yep, that's where I got the correct syntax. This is different than what you have in your readme which was the first confusing part. The new style syntax also doesn't like this at all. This is also the root source of the problem when using new style with your export plugin. There is a root POM-only repository at /com/lazan which points to the JAR repository at /gradle/plugin/com/lazan. The export plugin doesn't seem to export the POM-only dependency, breaking the use of new-style syntax with your plugin.
I am trying my entire process with the old-style for now.
This did not work either. The problem is that the JAR files under /gradle/plugin/ in your export do not have an associated POM file in the export.
+---gradle
| \---plugin
| \---com
| +---lazan
| | \---gradle-dependency-export
| | \---0.5
| | gradle-dependency-export-0.5.jar
| |
| \---palantir
| \---gradle
| \---gitversion
| \---gradle-git-version
| \---0.12.0-rc2
| gradle-git-version-0.12.0-rc2.jar
This breaks their use as a maven repo.
The POM files are available on the public server:
https://plugins.gradle.org/m2/gradle/plugin/com/lazan/gradle-dependency-export/0.5/gradle-dependency-export-0.5.pom
They're just not being exported by your plugin.
I wasn't aware that gradle plugins are this wierd thing where there's two poms and one jar.
I think I understand why the pom isn't exported. Configurations are a collection of dependencies which ultimately resolve to jars. Configurations do not point to poms. I had to hack that bit by integrating the Maven resolver with gradle but ultimately the starting point is the Gradle configuration.
I fear that the pom (for the pom only bit) is "lost" after its used in the Gradle internals and the configuration does not have a pointer to it.
I wonder if there's a way of querying the applied plugins to determine the coordinates (the GAV) to get these pom only artifacts.
Yes, it is very confusing. I'm not familiar with it myself, but looks to me almost like this is an adaptation to move forward with the new-style syntax while still supporting the old. In the old syntax, it looks like you had the luxury of specifying the group ID separately (gradle.plugin.com.lazan) where the new syntax assumes the package is available at the declared package name (com.lazan). So it almost looks like the one at com.lazan is a "pointer" (or a symlink in Unix terms) to the one published at the old-style syntax -- a POM-only pointing at a JAR+POM.
I'm okay with the old-style syntax, where it ignores the parent POM-only altogether. However, I still have the issue that the old style is not exporting the POM along with the JAR for those particular plugins. I'm not sure why that's the case since I'm pointing directly at the gradle.plugin.com.lazan and not the new style. This is also peculiar since I do not have this problem with the shadow plugin.
For the moment, you could possibly manually add the missing poms to a configuration so it's exported. I realise this is not a nice solution.
Eg:
buildscript {
configurations {
poms
}
dependencies {
poms 'gradle.plugin.com.lazan:gradle-dependency-export:0.5@pom'
}
}
The suggested syntax didn't work at all. You get:
Could not find method poms() for arguments [gradle.plugin.com.lazan:gradle-dependency-export:0.5@pom] on object of type org.gradle.api.internal.artifacts.dsl.dependencies.DefaultDependencyHandler.
It doesn't like the use of poms instead of classpath.
Wierd, try
buildscript {
configurations {
poms
}
dependencies.add('poms', 'gradle.plugin.com.lazan:gradle-dependency-export:0.5@pom')
}
Nope.
Could not run build action using Gradle distribution 'https://services.gradle.org/distributions/gradle-5.3.1-bin.zip'.
A problem occurred evaluating root project
Configuration with name 'poms' not found.
Seems like it doesn't like custom configurations inside the buildscript section?
That didn't work either. The problem is apparently worse than I thought. Even when I manually added those POM files to the repo, it pointed out other missing exports. For example, shadow apparently has a runtime dependency on org.codehaus.groovy:groovy-backports-compat23. This is not included in the export.
Major bummer... I'll try to apply/export shadow myself. Can you show me an example of how you load plugins from the folder? ie the "offline" usage of the exported folder.
On Wed, 17 Apr 2019, 11:48 pm robross0606, [email protected] wrote:
That didn't work either. The problem is apparently worse than I thought. Even when I manually added those POM files to the repo, it pointed out other missing exports. None of the transitive dependencies from those offending plugins made it into the export either. For example, shadow apparently has a dependency on org.codehaus.groovy:groovy-backports-compat that isn't expressed anywhere in anything. However, if you attempt to use this project where that package isn't available in a repository, it fails.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/uklance/gradle-dependency-export/issues/10#issuecomment-484289820, or mute the thread https://github.com/notifications/unsubscribe-auth/AAK3KJFX6QOBIUZG7XCZ4OTPQ6SK3ANCNFSM4HGXCUNQ .
I'm using this script: https://github.com/sonatype-nexus-community/nexus-repository-import-scripts to import your export into a disconnected (no Internet) Sonatype Nexus 3 repo.
The process itself is working fine. The Nexus repo shows all the resources that your plugin properly exported, as well as a few where I patched them with manually downloaded POM files to make them compatible.
The only problem I'm having is that some resources are still missing from the export so the resulting Nexus 3 server is ultimately missing some needed resources to make this project build from inside the disconnected lab.
We're scrambling to find some temporary alternative solutions for this issue. Do you happen to know of any method to do something similar directly with Maven? I'm aware of maven-dependency-plugin but not sure if that can export in standard Maven repo style like your plugin does. If you have any advice, I would very much appreciate it.
You could manually include all the missing plugin artifacts in a configuration (eg org.codehaus.groovy:groovy-backports-compat23). Hopefully there's not to many of these. Since plugins don't change very often this might be manageable. I think your compile/test/runtime dependencies are all exporting correctly so possibly this is a viable solution?
On Thu, 18 Apr 2019, 5:23 pm robross0606, [email protected] wrote:
We're scrambling to find some temporary alternative solutions for this issue. Do you happen to know of any method to do something similar directly with Maven? I'm aware of maven-dependency-plugin but not sure if that can export in standard Maven repo style like your plugin does. If you have any advice, I would very much appreciate it.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/uklance/gradle-dependency-export/issues/10#issuecomment-484580913, or mute the thread https://github.com/notifications/unsubscribe-auth/AAK3KJCOMAEO2SUVQZVFOKLPRCOA7ANCNFSM4HGXCUNQ .