Bouncy Castle Jar verification fails from a fat Jar under Oracle Java 17

[thelateperseus]

When running a Spring Boot app as a fat Jar under Java 17, using the Bouncy Castle provider results in an exception SecurityException: JCE cannot authenticate the provider BC with cause IllegalStateException: zip file closed. Any use of the provider seems to trigger the exception, e.g.

Cipher.getInstance("AES/CBC/PKCS5Padding","BC");

I have created a sample Spring Boot app that reproduces the problem.

I stepped through the code and I believe the problem is caused by the Spring Boot JarURLConnection returning an already closed Jar file from getJarFile(). I think this relates to issues #17127 and #25538, but I could be wrong.

This same issue does not occur under Java 11, so I assume something has changed in JarVerifier.verifySingleJar between Java 11 and 17.

The exception stack trace is:

Exception in thread "main" java.lang.reflect.InvocationTargetException
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:568)
        at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49)
        at org.springframework.boot.loader.Launcher.launch(Launcher.java:108)
        at org.springframework.boot.loader.Launcher.launch(Launcher.java:58)
        at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:88)
Caused by: java.lang.SecurityException: JCE cannot authenticate the provider BC
        at java.base/javax.crypto.Cipher.getInstance(Cipher.java:722)
        at java.base/javax.crypto.Cipher.getInstance(Cipher.java:642)
        at com.example.bctest.BctestApplication.main(BctestApplication.java:14)
        ... 8 more
Caused by: java.lang.IllegalStateException: zip file closed
        at java.base/java.util.zip.ZipFile.ensureOpen(ZipFile.java:831)
        at java.base/java.util.zip.ZipFile.getManifestName(ZipFile.java:1057)
        at java.base/java.util.zip.ZipFile$1.getManifestName(ZipFile.java:1100)
        at java.base/javax.crypto.JarVerifier.verifySingleJar(JarVerifier.java:461)
        at java.base/javax.crypto.JarVerifier.verifyJars(JarVerifier.java:317)
        at java.base/javax.crypto.JarVerifier.verify(JarVerifier.java:260)
        at java.base/javax.crypto.ProviderVerifier.verify(ProviderVerifier.java:130)
        at java.base/javax.crypto.JceSecurity.verifyProvider(JceSecurity.java:190)
        at java.base/javax.crypto.JceSecurity.getVerificationResult(JceSecurity.java:218)
        at java.base/javax.crypto.Cipher.getInstance(Cipher.java:718)
        ... 10 more

[snicoll]

See #28157.

[mikegike]

#28150 The ticket doesn’t consider the used Java Version. When using OpenJDK, what often Linux does, there is no Signature check for used provider jars. Oracle JDK checks the Signature for used providers.

See: https://www.baeldung.com/oracle-jdk-vs-openjdk Oracle has always required third party cryptographic providers to be signed by a known certificate, while cryptography framework in OpenJDK has an open cryptographic interface, which means there is no restriction as to which providers can be used

I think that is why there was a problem to recreate the issue.

[thelateperseus]

I can reproduce the issue with my test app (Spring Boot 2.5.7) on all the following configurations. Note that these are all different physical computers - WSL is running on a different Windows 10 machine.

• Oracle JDK 17.0.1 on Windows 10
• Oracle JDK 17.0.1 on Unubtu 20.04 (Windows Subsystem for Linux on Windows 10)
• Oracle JDK 17.0.1 on Windows Server 2016

However, the problem does not occur using OpenJDK 17.0.0 on Ubuntu 20.04 (Windows Subsystem for Linux).

Issue #28157 has a similar exception message, but a different root cause exception. I can reproduce my issue across multiple machines and operating systems, and the root cause is always "zip file closed." Each of these machines has downloaded a separate copy of the bouncy castle jar, so I don't believe it's corrupt.

[wilkinsona]

Thanks for the sample, @thelateperseus. I've reproduced the problem on macOS using Oracle JDK 17.0.1:

$ java -version
java version "17.0.1" 2021-10-19 LTS
Java(TM) SE Runtime Environment (build 17.0.1+12-LTS-39)
Java HotSpot(TM) 64-Bit Server VM (build 17.0.1+12-LTS-39, mixed mode, sharing)
$ java -jar build/libs/spring-boot-bouncy-castle-0.0.1-SNAPSHOT.jar 
Exception in thread "main" java.lang.reflect.InvocationTargetException
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49)
	at org.springframework.boot.loader.Launcher.launch(Launcher.java:108)
	at org.springframework.boot.loader.Launcher.launch(Launcher.java:58)
	at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:88)
Caused by: java.lang.SecurityException: JCE cannot authenticate the provider BC
	at java.base/javax.crypto.Cipher.getInstance(Cipher.java:722)
	at java.base/javax.crypto.Cipher.getInstance(Cipher.java:642)
	at com.example.bctest.BctestApplication.main(BctestApplication.java:14)
	... 8 more
Caused by: java.lang.IllegalStateException: zip file closed
	at java.base/java.util.zip.ZipFile.ensureOpen(ZipFile.java:831)
	at java.base/java.util.zip.ZipFile.getManifestName(ZipFile.java:1057)
	at java.base/java.util.zip.ZipFile$1.getManifestName(ZipFile.java:1100)
	at java.base/javax.crypto.JarVerifier.verifySingleJar(JarVerifier.java:461)
	at java.base/javax.crypto.JarVerifier.verifyJars(JarVerifier.java:317)
	at java.base/javax.crypto.JarVerifier.verify(JarVerifier.java:260)
	at java.base/javax.crypto.ProviderVerifier.verify(ProviderVerifier.java:130)
	at java.base/javax.crypto.JceSecurity.verifyProvider(JceSecurity.java:190)
	at java.base/javax.crypto.JceSecurity.getVerificationResult(JceSecurity.java:218)
	at java.base/javax.crypto.Cipher.getInstance(Cipher.java:718)
	... 10 more

[philwebb]

😭

[philwebb]

I've managed to find some time to dig into this today and unfortunately we have more than just the "zip file closed" issue to solve. I patched a local build so that close() is no longer called early and we get a different exception.

Exception in thread "main" java.lang.reflect.InvocationTargetException
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49)
	at org.springframework.boot.loader.Launcher.launch(Launcher.java:108)
	at org.springframework.boot.loader.Launcher.launch(Launcher.java:58)
	at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:88)
Caused by: java.lang.SecurityException: JCE cannot authenticate the provider BC
	at java.base/javax.crypto.Cipher.getInstance(Cipher.java:722)
	at java.base/javax.crypto.Cipher.getInstance(Cipher.java:642)
	at com.example.bctest.BctestApplication.main(BctestApplication.java:14)
	... 8 more
Caused by: java.util.jar.JarException: The JCE Provider jar:file:/Volumes/Data/projects/spring-boot/samples/spring-boot-bouncy-castle/build/libs/spring-boot-bouncy-castle-0.0.1-SNAPSHOT.jar!/BOOT-INF/lib/bcprov-jdk15on-1.69.jar!/ is not signed.
	at java.base/javax.crypto.JarVerifier.verifySingleJar(JarVerifier.java:464)
	at java.base/javax.crypto.JarVerifier.verifyJars(JarVerifier.java:317)
	at java.base/javax.crypto.JarVerifier.verify(JarVerifier.java:260)
	at java.base/javax.crypto.ProviderVerifier.verify(ProviderVerifier.java:130)
	at java.base/javax.crypto.JceSecurity.verifyProvider(JceSecurity.java:190)
	at java.base/javax.crypto.JceSecurity.getVerificationResult(JceSecurity.java:218)
	at java.base/javax.crypto.Cipher.getInstance(Cipher.java:718)
	... 10 more

It took a bit more digging to get to the bottom of it (not helped by the lack of source code for JarVerifier). I think what happens is verifySingleJar is trying to check if the nested bcprov-jdk15on-1.69.jar jar is signed. To do that, it has the following code (more or less):

if (!jarManifestNameChecked && SharedSecrets.getJavaUtilZipFileAccess().getManifestName(jf, true) == null) {
    throw new JarException("The JCE Provider " + jarURL.toString() + " is not signed.");
}

The SharedSecrets.getJavaUtilZipFileAccess().getManifestName(jf, true) call ends up calling ZipFile.getManifestName(onlyIfSignatureRelatedFiles). Unfortunately this is a private method so our JarFile implementation can't override it. This means that rather than checking the nested jar, we end up checking the root jar. Since there are not signature files in the root jar the getManifestName method returns null.

I've managed to hack around the problem by adding an empty META-INF/BOOT.SF file in the fat jar. This is enough to get getManifestName to return a non null value and things then start up fine.

I'm not sure that this is really a good long term suggestions. I'll flag the issue for team attention to see if anyone has any bright ideas. Ideally we'd like getManifestName to be a protected method, then we could override it.

[philwebb]

Hacked up code is at https://github.com/philwebb/spring-boot/tree/gh-28837

[zezulka000]

I am having same issue on spring boot 2.6.2 and oracle java 17 (oracle java 11 is working, same as open JDK versions as listed above). I excluded dependency to bouncy castle jar, and added external classpath to fat jar, but problem remained the same: java.lang.SecurityException: JCE cannot authenticate the provider BC Caused by: java.lang.IllegalStateException: zip file closed

[zeroleak]

same blocking issue here

[evgenyigumnov]

I have the same blocking issue :(

[snicoll]

@zeroleak @evgenyigumnov sorry about that but please use the reaction on the original description rather than this.

[evgenyigumnov]

@zeroleak @evgenyigumnov sorry about that but please use the reaction on the original description rather than this.

@snicoll what do you mean? I can not use Oracle JDK 11 version. I have to use 17 version.

[snicoll]

I mean that "I have the same issue" is not very helpful and send a notification to the 3K watchers of this repository. If you want to indicate you're affected by this issue without any extra information, please rather use the reaction (👍) in the original description above.

[varkychen]

I could work around this issue by running my Spring Boot app using an exploded directory format. This is the recommended approach by Spring Boot for container images → Container Images

[philwebb]

For anyone watching this issue I just pushed a fix for #29356 which should allow <requiresUnpack> to work.

[thelateperseus]

I've tested this with my sample Spring Boot app and it works now. I had to change the Spring Boot version in build.gradle to 2.6.9 and add this section to the end of the file:

bootJar {
    requiresUnpack '**/bcprov-jdk15on-*.jar'
}

[mikegike]

Works also for me using maven.

<plugins>
  <plugin>
   <groupId>org.springframework.boot</groupId>
   <artifactId>spring-boot-maven-plugin</artifactId>
    <configuration>
    <requiresUnpack>
     <dependency>
       <groupId>org.bouncycastle</groupId>
       <artifactId>bcprov-jdk15on</artifactId>
     </dependency>
   </requiresUnpack>
   </configuration>
 </plugin>
<plugins>

With one exception, when instantiating several times the bouncy castle provider using new BouncyCastleProvider(), in different context, still an exception come up. Changing to "BC" solve the issue. Example:

public static void main(String[] args) {
    Security.addProvider(new BouncyCastleProvider());
    
    try {
      
      MessageDigest md = MessageDigest.getInstance("SHA-256", new BouncyCastleProvider()); 
      byte[] iv = SecureRandom.getInstanceStrong().generateSeed(16);
      byte[] keyBytes = md.digest("test".getBytes());
      SecretKey key = new SecretKeySpec(keyBytes, "AES"); 
      
      Cipher cipher = Cipher.getInstance("AES/CBC/PKCS5Padding", new BouncyCastleProvider());
      cipher.init(Cipher.ENCRYPT_MODE, key, new IvParameterSpec(iv));
      
      
      KeyStore pkcs12KeyStore = KeyStore.getInstance("PKCS12", new BouncyCastleProvider());
      pkcs12KeyStore.load(new ByteArrayInputStream(Base64.decode(teststore)), "hoaghoag".toCharArray());
    } catch (Exception e) {
      e.printStackTrace();
    }
    SpringApplication.run(BcprovidertestApplication.class, args);
  }

ends up in:

"C:\develop\Java\Oracle\jdk-17.0.2\bin\java.exe" -jar .\bcprovidertest-0.0.1-SNAPSHOT.jar
java.io.IOException: error constructing MAC: java.lang.SecurityException: JCE cannot authenticate the provider BC
        at org.bouncycastle.jcajce.provider.keystore.pkcs12.PKCS12KeyStoreSpi.engineLoad(Unknown Source)
        at org.bouncycastle.jcajce.provider.keystore.util.AdaptingKeyStoreSpi.engineLoad(Unknown Source)
        at java.base/java.security.KeyStore.load(KeyStore.java:1473)
        at mis.bcprovidertest.BcprovidertestApplication.main(BcprovidertestApplication.java:53)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:568)
        at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49)
        at org.springframework.boot.loader.Launcher.launch(Launcher.java:108)
        at org.springframework.boot.loader.Launcher.launch(Launcher.java:58)
        at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:88)

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::                (v2.6.9)

2022-07-01 15:25:13.136  INFO 7724 --- [           main] m.b.BcprovidertestApplication            : Starting BcprovidertestApplication v0.0.1-SNAPSHOT using Java 17.0.2 on HO1500307 with PID 7724 (C:\develop\eclipse\eclipse2018\workspace\bcprovidertest\target\bcprovidertest-0.0.1-SNAPSHOT.jar started by m.scholl in C:\develop\eclipse\eclipse2018\workspace\bcprovidertest\target)
2022-07-01 15:25:13.139  INFO 7724 --- [           main] m.b.BcprovidertestApplication            : No active profile set, falling back to 1 default profile: "default"
2022-07-01 15:25:13.652  INFO 7724 --- [           main] m.b.BcprovidertestApplication            : Started BcprovidertestApplication in 0.933 seconds (JVM running for 2.704)

changing to the following code, everything works fine:

public static void main(String[] args) {
    Security.addProvider(new BouncyCastleProvider());
    
    try {

      MessageDigest md = MessageDigest.getInstance("SHA-256", "BC"); 
      byte[] iv = SecureRandom.getInstanceStrong().generateSeed(16);
      byte[] keyBytes = md.digest("test".getBytes());
      SecretKey key = new SecretKeySpec(keyBytes, "AES"); 

      Cipher cipher = Cipher.getInstance("AES/CBC/PKCS5Padding", "BC");
      cipher.init(Cipher.ENCRYPT_MODE, key, new IvParameterSpec(iv));
      
      KeyStore pkcs12KeyStore = KeyStore.getInstance("PKCS12", "BC");
      pkcs12KeyStore.load(new ByteArrayInputStream(Base64.decode(teststore)), "hoaghoag".toCharArray());
    } catch (Exception e) {
      // TODO Auto-generated catch block
      e.printStackTrace();
    }

    SpringApplication.run(BcprovidertestApplication.class, args);
  }

[philwebb]

For anyone watching this issue I just pushed a fix for #29356 which should allow <requiresUnpack> to work.

Unfortunately the fix caused regressions so we're going to need to revert it. We'll have another go in #31853

[FULaBUla]

I've tested this with my sample Spring Boot app and it works now. I had to change the Spring Boot version in build.gradle to 2.6.9 and add this section to the end of the file:

bootJar {
    requiresUnpack '**/bcprov-jdk15on-*.jar'
}

2.7.3 and 2.7.4, requiresUnpack doesn't work >_<

[snicoll]

@FULaBUla please read the comment just above yours. We had to revert the fix as it was causing regressions.

[q1045243113]

have any temporary solutions?

[just-victor]

Still no solution?

[epca-job]

@philwebb Has this BUG been forgotten？It's been a long time.

[gxb1t]

executing ./jarsigner -verify my-project/BOOT-INF/lib/bcprov-jdk15on-1.68.jar -J-Djava.security.debug=jar -verbose

I get this message

Warning: This jar contains entries whose certificate chain is invalid. Reason: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target The DSA signing key has a keysize of 1024 which is considered a security risk. This key size will be disabled in a future update.

 

The signer certificate expired on 2022-03-11. However, the JAR will be valid until the timestamp expires on 2030-10-17.

I don't know if it's a problem of certificate but at the end I solved changing the provider from BC to SUN.

Provider defaultProvider = Security.getProvider("SUN");
Security.addProvider(defaultProvider);

and then

KeyStore keyStore = KeyStore.getInstance("PKCS12", "SUN");

[philwebb]

Has this BUG been forgotten？It's been a long time.

No it's not been forgotten, we just don't have an easy way to fix it and we have limited bandwidth.

[JavaLionLi]

springboot 3.0.6 + oracle 17.0.6 JCE cannot authenticate the provider BC Still have this problem I don't know when it's going to fix it.

[snicoll]

@JavaLionLi the answer is right above your comment.

[JavaLionLi]

@JavaLionLi the answer is right above your comment.

Using the following configuration has no effect

<plugins>
  <plugin>
   <groupId>org.springframework.boot</groupId>
   <artifactId>spring-boot-maven-plugin</artifactId>
    <configuration>
      <requiresUnpack>
       <dependency>
         <groupId>org.bouncycastle</groupId>
         <artifactId>bcprov-jdk15on</artifactId>
       </dependency>
     </requiresUnpack>
   </configuration>
 </plugin>
<plugins>

[snicoll]

Please take the time to read the history before commenting. See https://github.com/spring-projects/spring-boot/issues/28837#issuecomment-1218363870

[JavaLionLi]

Please take the time to read the history before commenting. See #28837 (comment)

I looked at it, but I couldn't find how to solve the problem All the methods mentioned above have been tried