plugin-installation-manager-tool icon indicating copy to clipboard operation
plugin-installation-manager-tool copied to clipboard

Published 20 hours ago • verified publisher icon jenkinsci

Some features slightly fragile

Open pwillis-els opened this issue 4 years ago • 9 comments

Given the following plugins.txt file:

active-directory
aws-parameter-store
configuration-as-code
configuration-as-code-secret-ssm
git
github
job-dsl
ldap
role-strategy
timestamper

The following command returns an error:

docker run \
    --rm -it \
    -v `pwd`/plugins.txt:/app/plugins.txt \
    jenkins/jenkins \
        jenkins-plugin-cli \
            --plugin-file /app/plugins.txt \
            --view-all-security-warnings \
            --no-download
java.util.ArrayList cannot be cast to java.lang.Comparable

This seems to be a bug, can anyone confirm?

If the plugins.txt contains git:4.1.9, a different command throws an error when it cannot resolve a version dependency:

docker run \
    --rm -it \
    -v `pwd`/plugins.txt:/app/plugins.txt \
    jenkins/jenkins \
        jenkins-plugin-cli \
            --plugin-file /app/plugins.txt \
            --view-security-warnings \
            --no-download \
            --skip-failed-plugins
Downloaded file is not a valid ZIP
Unable to resolve dependencies for git
Plugin github:1.32.0 depends on git:4.5.1, but there is an older version defined on the top level - git:4.1.9

For this second error, it would be useful if there were a "--force" or "--ignore-errors" option. The "--skip-failed-plugins" option doesn't seem to change the result.

pwillis-els avatar Dec 27 '20 18:12 pwillis-els

first one is clearly a bug.

second one is not a bug, that is a design choice. Your plugins.txt is meant to pin dependencies. You could try --latest-specified

jetersen avatar Dec 27 '20 18:12 jetersen

Thanks for the reply. Unfortunately --latest-specified seems to result in same behavior.

This is fine for the default, but without a way to ignore this error it's not easy to scan an existing plugins.txt file for vulnerable plugins. To work around it i'd need to create a new script to only run on on plugin one at a time, which kinda defeats the purpose of the existing tool :) I ended up doing that, but it took 90 seconds to iterate through the above plugins. The benefit here is doing them individually does not result in errors due to avoiding the dependency checking, but it does find vulnerable versions.

To work around the issue for the moment I wrote the following script and added some functions to help with the immediate task I'm trying to accomplish (check for any plugins with a known security vuln and bump them to the last known secure version): https://gist.github.com/pwillis-els/369f140b1db1efe13d2b609d54579ea6

pwillis-els avatar Dec 27 '20 21:12 pwillis-els

What prevents you from bumping to minimum required versions? Jenkins would fail to load the dependencies if you did not resolve the issue anyhow.

jetersen avatar Dec 27 '20 21:12 jetersen

I am able to replicate the first bug you reported inside unit testing.

java.lang.ClassCastException: class java.util.ArrayList cannot be cast to class java.lang.Comparable (java.util.ArrayList and java.lang.Comparable are in module java.base of loader 'bootstrap')

	at java.base/java.util.Comparators$NaturalOrderComparator.compare(Comparators.java:47)
	at java.base/java.util.TimSort.countRunAndMakeAscending(TimSort.java:355)
	at java.base/java.util.TimSort.sort(TimSort.java:234)
	at java.base/java.util.Arrays.sort(Arrays.java:1515)
	at java.base/java.util.stream.SortedOps$SizedRefSortingSink.end(SortedOps.java:353)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:485)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
	at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497)
	at io.jenkins.tools.pluginmanager.impl.PluginManager.showAllSecurityWarnings(PluginManager.java:353)
	at io.jenkins.tools.pluginmanager.impl.PluginManager.start(PluginManager.java:162)
	at io.jenkins.tools.pluginmanager.impl.PluginManager.start(PluginManager.java:125)
	at io.jenkins.tools.pluginmanager.impl.PluginManagerIntegrationTest.debugArrayListComparable(PluginManagerIntegrationTest.java:292)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
	at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
	at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
	at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
	at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:69)
	at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:33)
	at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:220)
	at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:53)

jetersen avatar Dec 27 '20 22:12 jetersen

I figured it was better not to assume the behavior of other components outside of the immediate plugin functionality. For example, say you have an existing jenkins install with a bunch of plugins, and somebody updates a plugins.txt with a new invalid version. If you try to install that updated plugins.txt, jenkins may fail to install them, but it will also keep operating with the existing plugins. If the user has a separate task which checks the plugins.txt for vulnerable versions, the user may want that task to succeed and identify the vulnerable plugin versions, even if it's not installable yet.

In any case I think I got what you mean, using these options: --available-updates --no-download --plugins $(cat plugins.txt). For a set of pinned versions, that may give me the following:

Available updates:
active-directory (2.18) has an available update: 2.23
configuration-as-code (1.27) has an available update: 1.46
git (4.2.1) has an available update: 4.5.1
github (1.29.2) has an available update: 1.32.0
job-dsl (1.72) has an available update: 1.77
timestamper (1.11.2) has an available update: 1.11.8

But actually these seem to be the latest versions rather than the minimum secure versions. If you go through all the warnings and dependencies you'll find that, for example, you could use active-directory 2.20 instead of 2.23 here. I only want to fix the security vulns, I don't want to update past that point to the latest versions (as dependency management then drags a bunch of other deps along and then you're updating 8 plugins rather than 1)

pwillis-els avatar Dec 27 '20 22:12 pwillis-els

latest is the default you can use --latest false

jetersen avatar Dec 27 '20 22:12 jetersen

Unfortunately --latest false doesn't change the behavior.

It also seems like you can see the available updates, but you get an error on security warnings:

$ cat plugins.txt
active-directory
aws-parameter-store
configuration-as-code
configuration-as-code-secret-ssm
git:4.1.9
github
job-dsl
ldap
role-strategy
timestamper

$ docker run --rm jenkins/jenkins jenkins-plugin-cli  --available-updates --no-download --plugins `cat plugins.txt`
Available updates:
git (4.1.9) has an available update: 4.5.1

$ docker run --rm jenkins/jenkins jenkins-plugin-cli  --view-security-warnings --no-download --plugins `cat plugins.txt`
Downloaded file is not a valid ZIP
Unable to resolve dependencies for git
Plugin github:1.32.0 depends on git:4.5.1, but there is an older version defined on the top level - git:4.1.9

pwillis-els avatar Dec 27 '20 22:12 pwillis-els

on the last command what does it print when you add --verbose

jetersen avatar Dec 27 '20 22:12 jetersen

this is the stacktrace. So the Downloaded file is not a valid ZIP is sort of misleading. The download got stopped plugin is not compatible with the top level version.

java.nio.file.NoSuchFileException: C:\Users\joseph\AppData\Local\Temp\git15863940145146328763.jpi
	at java.base/sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:85)
	at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:103)
	at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:108)
	at java.base/sun.nio.fs.WindowsFileAttributeViews$Basic.readAttributes(WindowsFileAttributeViews.java:53)
	at java.base/sun.nio.fs.WindowsFileAttributeViews$Basic.readAttributes(WindowsFileAttributeViews.java:38)
	at java.base/sun.nio.fs.WindowsFileSystemProvider.readAttributes(WindowsFileSystemProvider.java:198)
	at java.base/java.nio.file.Files.readAttributes(Files.java:1763)
	at java.base/java.util.zip.ZipFile$Source.get(ZipFile.java:1225)
	at java.base/java.util.zip.ZipFile$CleanableResource.<init>(ZipFile.java:727)
	at java.base/java.util.zip.ZipFile$CleanableResource.get(ZipFile.java:844)
	at java.base/java.util.zip.ZipFile.<init>(ZipFile.java:247)
	at java.base/java.util.zip.ZipFile.<init>(ZipFile.java:177)
	at java.base/java.util.jar.JarFile.<init>(JarFile.java:348)
	at java.base/java.util.jar.JarFile.<init>(JarFile.java:319)
	at java.base/java.util.jar.JarFile.<init>(JarFile.java:285)
	at io.jenkins.tools.pluginmanager.impl.PluginManager.downloadToFile(PluginManager.java:1145)
	at io.jenkins.tools.pluginmanager.impl.PluginManager.downloadToFile(PluginManager.java:1073)
	at io.jenkins.tools.pluginmanager.impl.PluginManager.downloadPlugin(PluginManager.java:1002)
	at io.jenkins.tools.pluginmanager.impl.PluginManager.resolveDependenciesFromManifest(PluginManager.java:788)
	at io.jenkins.tools.pluginmanager.impl.PluginManager.resolveDirectDependencies(PluginManager.java:913)
	at io.jenkins.tools.pluginmanager.impl.PluginManager.resolveRecursiveDependencies(PluginManager.java:943)
	at io.jenkins.tools.pluginmanager.impl.PluginManager.findPluginsAndDependencies(PluginManager.java:570)
	at io.jenkins.tools.pluginmanager.impl.PluginManager.start(PluginManager.java:167)
	at io.jenkins.tools.pluginmanager.impl.PluginManager.start(PluginManager.java:126)
	at io.jenkins.tools.pluginmanager.cli.Main.main(Main.java:70)
java.nio.file.NoSuchFileException: C:\Users\joseph\AppData\Local\Temp\git15863940145146328763.jpi
	at java.base/sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:85)
	at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:103)
	at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:108)
	at java.base/sun.nio.fs.WindowsFileSystemProvider.implDelete(WindowsFileSystemProvider.java:274)
	at java.base/sun.nio.fs.AbstractFileSystemProvider.delete(AbstractFileSystemProvider.java:105)
	at java.base/java.nio.file.Files.delete(Files.java:1141)
	at io.jenkins.tools.pluginmanager.impl.PluginManager.resolveDependenciesFromManifest(PluginManager.java:789)
	at io.jenkins.tools.pluginmanager.impl.PluginManager.resolveDirectDependencies(PluginManager.java:913)
	at io.jenkins.tools.pluginmanager.impl.PluginManager.resolveRecursiveDependencies(PluginManager.java:943)
	at io.jenkins.tools.pluginmanager.impl.PluginManager.findPluginsAndDependencies(PluginManager.java:570)
	at io.jenkins.tools.pluginmanager.impl.PluginManager.start(PluginManager.java:167)
	at io.jenkins.tools.pluginmanager.impl.PluginManager.start(PluginManager.java:126)
	at io.jenkins.tools.pluginmanager.cli.Main.main(Main.java:70)
io.jenkins.tools.pluginmanager.impl.PluginDependencyStrategyException: Plugin github:1.32.0 depends on git:4.5.1, but there is an older version defined on the top level - git:4.1.9
	at io.jenkins.tools.pluginmanager.impl.PluginManager.resolveRecursiveDependencies(PluginManager.java:961)
	at io.jenkins.tools.pluginmanager.impl.PluginManager.findPluginsAndDependencies(PluginManager.java:570)
	at io.jenkins.tools.pluginmanager.impl.PluginManager.start(PluginManager.java:167)
	at io.jenkins.tools.pluginmanager.impl.PluginManager.start(PluginManager.java:126)
	at io.jenkins.tools.pluginmanager.cli.Main.main(Main.java:70)
Plugin github:1.32.0 depends on git:4.5.1, but there is an older version defined on the top level - git:4.1.9

Process finished with exit code 1

jetersen avatar Dec 27 '20 23:12 jetersen