Classloader problem regarding dynamic resolving of BouncyCastle in Tomcat 9

[norbertroamsys]

We use the sshj library in our Java 11/Tomcat 9 environment and it works quite fine! But after a redeploy of the web application in Tomcat we ran into the following classloader problem reported by the Tomcat Webapp classloader:

20-Mar-2023 12:07:51.307 SEVERE [sshj-Reader-apollo/10.10.0.5:22-1679310471302] net.schmizz.sshj.transport.TransportImpl.die Dying because - class configured for KeyPairGenerator (provider: BC) cannot be found.
        net.schmizz.sshj.common.SSHRuntimeException: class configured for KeyPairGenerator (provider: BC) cannot be found.
                at net.schmizz.sshj.transport.kex.DHBase.<init>(DHBase.java:41)
                at net.schmizz.sshj.transport.kex.Curve25519DH.<init>(Curve25519DH.java:35)
                at net.schmizz.sshj.transport.kex.Curve25519SHA256.<init>(Curve25519SHA256.java:54)
                at net.schmizz.sshj.transport.kex.Curve25519SHA256$Factory.create(Curve25519SHA256.java:44)
                at net.schmizz.sshj.transport.kex.Curve25519SHA256$Factory.create(Curve25519SHA256.java:39)
                at net.schmizz.sshj.common.Factory$Named$Util.create(Factory.java:54)
                at net.schmizz.sshj.transport.KeyExchanger.gotKexInit(KeyExchanger.java:242)
                at net.schmizz.sshj.transport.KeyExchanger.handle(KeyExchanger.java:380)
                at net.schmizz.sshj.transport.TransportImpl.handle(TransportImpl.java:495)
                at net.schmizz.sshj.transport.Decoder.decode(Decoder.java:113)
                at net.schmizz.sshj.transport.Decoder.received(Decoder.java:200)
                at net.schmizz.sshj.transport.Reader.run(Reader.java:60)
        Caused by: java.security.NoSuchAlgorithmException: class configured for KeyPairGenerator (provider: BC) cannot be found.
                at java.base/java.security.Provider$Service.getImplClass(Provider.java:1863)
                at java.base/java.security.Provider$Service.newInstance(Provider.java:1824)
                at java.base/sun.security.jca.GetInstance.getInstance(GetInstance.java:236)
                at java.base/sun.security.jca.GetInstance.getInstance(GetInstance.java:206)
                at java.base/java.security.KeyPairGenerator.getInstance(KeyPairGenerator.java:300)
                at net.schmizz.sshj.common.SecurityUtils.getKeyPairGenerator(SecurityUtils.java:188)
                at net.schmizz.sshj.transport.kex.DHBase.<init>(DHBase.java:38)
                ... 11 more
        Caused by: java.lang.ClassNotFoundException: Illegal access: this web application instance has been stopped already. Could not load [org.bouncycastle.jcajce.provider.asymmetric.ec.KeyPairGeneratorSpi$ECDSA]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access.
                at org.apache.catalina.loader.WebappClassLoaderBase.checkStateForClassLoading(WebappClassLoaderBase.java:1375)
                at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1226)
                at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1188)
                at java.base/java.security.Provider$Service.getImplClass(Provider.java:1850)
                ... 17 more
        Caused by: java.lang.IllegalStateException: Illegal access: this web application instance has been stopped already. Could not load [org.bouncycastle.jcajce.provider.asymmetric.ec.KeyPairGeneratorSpi$ECDSA]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access.
                at org.apache.catalina.loader.WebappClassLoaderBase.checkStateForResourceLoading(WebappClassLoaderBase.java:1385)
                at org.apache.catalina.loader.WebappClassLoaderBase.checkStateForClassLoading(WebappClassLoaderBase.java:1373)
                ... 20 more
20-Mar-2023 12:07:51.320 SEVERE [http-nio-8084-exec-18] net.schmizz.concurrent.Promise.tryRetrieve <<kex done>> woke to: net.schmizz.sshj.transport.TransportException: class configured for KeyPairGenerator (provider: BC) cannot be found.

We found some information in Stack Overflow, but no acceptable solution.

The reason for this behavior is the current implementation in SecurityUtils:

• registerSecurityProvider(className) resolves the Class by Class.forName(className) using the calling class classloader
• The found provider is registered in JDK's registry by Security.addProvider(provider)
• After a redeploy this provider (or it's Class) is invalid because it has been loaded by a class loader no longer exist.

We are not sure how to fix the problem in an acceptable way.

What we do as a workaround is cleaning JDK's Security using the following code in a static initializer of our SSHClient factory:

    static {
        /*
         * Remove provider that has been registered in SecurityUtils with different class loader
         * to avoid: "Illegal access: this web application instance has been stopped already. Could not load [org.bouncycastle..."
         */
        Security.removeProvider(SecurityUtils.BOUNCY_CASTLE);
    }

This is ok in our Use Case because SSHJ is the only library that makes usage of BouncyCastle provider.

A solution that should work maybe an addition setter in SecurityUtils:

    /**
     * Set the JCE security provider that should be used.
     *
     * @param provider security provider instance
     */
    public static synchronized void setProvider(Provider provider) {
            if (Security.getProvider(provider.getName()) != null) {
                Security.removeProvider(provider.getName());
            }
            Security.addProvider(provider);
            SecurityUtils.securityProvider = provider.getName();  
            registerBouncyCastle = false;
            registrationDone = true;
    } 

Than the using code is able to set the provider from outside: SecurityUtils.setProvider(new BouncyCastleProvider());

The using code has to define a direct dependency to the bouncy castle lib instead of using resolving at runtime by Class.forName().

Maybe the issue https://github.com/hierynomus/sshj/issues/782 is somehow related to this and the code change suggested can solve both problems?

Used version:

        <dependency>
            <groupId>com.hierynomus</groupId>
            <artifactId>sshj</artifactId>
            <version>0.35.0</version>
        </dependency>

Thanks for your assistants! If our solution is acceptable for you we can also provide a pull request for changing the code ;-).

[rekantlo]

We are facing a similar problem while using java 8/Tomcat 9. We have been able to get around this by restarting production tomcats but users and us are looking for a more permanent solution. Here is our error.

2024-04-22 10:06:49,561 ERROR - [https-jsse-nio-7010-exec-421] concurrent.Promise (Promise.java:174) - [<kex done]> woke to: 
net.schmizz.sshj.transport.TransportException: class configured for KeyPairGenerator (provider: BC) cannot be found.

2024-04-22 10:06:49,554 ERROR - [reader] transport.TransportImpl (TransportImpl.java:612) - Dying because - class configured for
KeyPairGenerator (provider: BC) cannot be found. net.schmizz.sshj.common.SSHRuntimeException: class configured for
KeyPairGenerator (provider: BC) cannot be found. at net.schmizz.sshj.transport.kex.DHBase.[init](DHBase.java:41) ~[sshj-0.31.0.jar:0.31.0]

SSHJ version

<groupId>com.hierynomus</groupId>
            <artifactId>sshj</artifactId>
            <version>0.31.0</version>

This seems to only happen when using SFTP.

Any tips on how to fix this or a permanent solution would be greatly appreciated.