Handshake failed despite cipher being present and protocol being enabled

[GeluOltean]

Hey everyone. I recently ran into an issue when switching from using Java's naming directory to implementing LDAP integration via the UnboundID ldapsdk. I've extracted the parts that seem to throw the exception into a separate class and can provide it if required, so please let me know. Your help is greatly appreciated.

Specs

OS: Ubuntu 20.04.2 LTS x86_64 JDK version: Adopt OpenJDK 8.0.272 Hotspot TLS 1 and 1.1 via OpenSSL manually enabled system-wide UnboundID LDAPsdk version: 4.0.11, 4.0.13 and 6.0.0

Description

TLS handshake fails consistently when trying to connect to a server with the following protocol and ciphers. Details via OpenSSL:

---
No client certificate CA names sent
Peer signing digest: MD5-SHA1
Peer signature type: RSA
Server Temp Key: ECDH, P-256, 256 bits
---
SSL handshake has read 5328 bytes and written 280 bytes
Verification: OK
---
New, TLSv1.0, Cipher is ECDHE-RSA-AES128-SHA
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1
    Cipher    : ECDHE-RSA-AES128-SHA
    Session-ID: <ID>
    Session-ID-ctx: 
    Master-Key: <KEY>
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: 1626243347
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
    Extended master secret: yes

Besides the default enabled cipher suites, I added some additional ones, as well as older protocols. The code is Scala, but it's descriptive (if you'd like, I can convert it to Java).

val ciphers = Set(
    "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256",
    "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256",
    "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA",
    "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA",
    "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256",
    "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256",
    "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256",
    "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256",
    "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA",
    "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA",
    "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256",
    "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256",
    "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256",
    "TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384",
    "TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384",
    "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256",
    "TLS_DHE_DSS_WITH_AES_256_GCM_SHA384",
    "TLS_DHE_RSA_WITH_AES_256_GCM_SHA384",
    "TLS_DHE_DSS_WITH_AES_128_GCM_SHA256")
val suites = SSLUtil.getEnabledSSLCipherSuites
suites.addAll(ciphers.asJava)
SSLUtil.setEnabledSSLCipherSuites(suites)

SSLUtil.setEnabledSSLProtocols(List(SSLUtil.SSL_PROTOCOL_SSL_3, SSLUtil.SSL_PROTOCOL_TLS_1, SSLUtil.SSL_PROTOCOL_TLS_1_1, SSLUtil.SSL_PROTOCOL_TLS_1_2, SSLUtil.SSL_PROTOCOL_TLS_1_3).asJava)

Then, for the SSLUtil and LDAPConnection instantiation:

val ssl = new SSLUtil(new TrustAllTrustManager())
val conn = new LDAPConnection(ssl.createSSLSocketFactory(SSLUtil.SSL_PROTOCOL_TLS_1_1), ldapUri.getHost, ldapUri.getPort, ldapPrincipal, ldapPass)
conn.bind(new SimpleBindRequest(ldapPrincipal, ldapPass))

This fails when instantiating the LDAPConnection with the following stack trace:

Exception in thread "main" LDAPException(resultCode=91 (connect error), errorMessage='An error occurred while attempting to connect to server <LDAPURL>:  IOException(LDAPException(resultCode=91 (connect error), errorMessage='An error occurred while attempting to establish a connection to server <LDAPURL>:  SSLHandshakeException(No appropriate protocol (protocol is disabled or cipher suites are inappropriate)), ldapSDKVersion=6.0.0, revision=524c20f3bbcc0d83fb56b9e136a2fd3a7f60437d'))')
	at com.unboundid.ldap.sdk.LDAPConnection.connect(LDAPConnection.java:915)
	at com.unboundid.ldap.sdk.LDAPConnection.connect(LDAPConnection.java:802)
	at com.unboundid.ldap.sdk.LDAPConnection.connect(LDAPConnection.java:740)
	at com.unboundid.ldap.sdk.LDAPConnection.<init>(LDAPConnection.java:560)
	at com.unboundid.ldap.sdk.LDAPConnection.<init>(LDAPConnection.java:696)
	at com.unboundid.ldap.sdk.LDAPConnection.<init>(LDAPConnection.java:658)
	at NewScratcher$.delayedEndpoint$NewScratcher$1(Scratcher.scala:144)
	at NewScratcher$delayedInit$body.apply(Scratcher.scala:102)
	at scala.Function0.apply$mcV$sp(Function0.scala:39)
	at scala.Function0.apply$mcV$sp$(Function0.scala:39)
	at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:17)
	at scala.App.$anonfun$main$1(App.scala:73)
	at scala.App.$anonfun$main$1$adapted(App.scala:73)
	at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:553)
	at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:551)
	at scala.collection.AbstractIterable.foreach(Iterable.scala:920)
	at scala.App.main(App.scala:73)
	at scala.App.main$(App.scala:71)
	at NewScratcher$.main(Scratcher.scala:102)
	at NewScratcher.main(Scratcher.scala)
Caused by: java.io.IOException: LDAPException(resultCode=91 (connect error), errorMessage='An error occurred while attempting to establish a connection to server <LDAPURL>:  SSLHandshakeException(No appropriate protocol (protocol is disabled or cipher suites are inappropriate)), ldapSDKVersion=6.0.0, revision=524c20f3bbcc0d83fb56b9e136a2fd3a7f60437d')
	at com.unboundid.ldap.sdk.LDAPConnectionInternals.<init>(LDAPConnectionInternals.java:204)
	at com.unboundid.ldap.sdk.LDAPConnection.connect(LDAPConnection.java:904)
	... 19 more
Caused by: LDAPException(resultCode=91 (connect error), errorMessage='An error occurred while attempting to establish a connection to server <LDAPURL>:  SSLHandshakeException(No appropriate protocol (protocol is disabled or cipher suites are inappropriate)), ldapSDKVersion=6.0.0, revision=524c20f3bbcc0d83fb56b9e136a2fd3a7f60437d')
	at com.unboundid.ldap.sdk.ConnectThread.getConnectedSocket(ConnectThread.java:287)
	at com.unboundid.ldap.sdk.LDAPConnectionInternals.<init>(LDAPConnectionInternals.java:185)
	... 20 more
Caused by: javax.net.ssl.SSLHandshakeException: No appropriate protocol (protocol is disabled or cipher suites are inappropriate)
	at sun.security.ssl.HandshakeContext.<init>(HandshakeContext.java:171)
	at sun.security.ssl.ClientHandshakeContext.<init>(ClientHandshakeContext.java:98)
	at sun.security.ssl.TransportContext.kickstart(TransportContext.java:220)
	at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:428)
	at com.unboundid.util.ssl.SetEnabledProtocolsAndCipherSuitesSocket.startHandshake(SetEnabledProtocolsAndCipherSuitesSocket.java:926)
	at com.unboundid.ldap.sdk.ConnectThread.run(ConnectThread.java:173)

Process finished with exit code 1

When starting up the app, I added some debug options for the VM, specifically -Djdk.tls.client.protocols="TLSv1.1" -Dhttps.protocols="TLSv1.1" -Djavax.net.debug=all. With the debugging options on, there's some more interesting bits which I've added to the log_ldap.txt file. Oddly enough the last print of com.unboundid.util.ssl.TLSCipherSuiteSelector Results: only includes TLSv1.2 and TLSv1.3, but when calling SSLUtil.getEnabledSSLProtocols the result includes SSLv3, TLSv1, TLSv1.1, TLSv1.2, TLSv1.3.

Please let me know if there is any other information that I can provide. log_ldap.txt

[dirmgr]

I’ve looked into this, but haven’t been able to reproduce the problem that you’re describing. I can write a simple Java program that uses TLSv1 or TLSv1.1 to interact with a server that only supports those protocol versions.

The 6.0.0 release of the LDAP SDK did disable support for TLSv1 and TLSv1.1 by default because those protocol versions are no longer considered secure or recommended as per RFC 8996. It also disabled support for TLS cipher suites that rely on the SHA-1 message digest algorithm (which is also no longer considered secure) and RSA key exchange (which doesn’t support forward secrecy). However, while we certainly don’t recommend using those TLS protocol versions or cipher suites, it should still be possible to explicitly enable support for them through the SSLUtil.setEnabledSSLProtocols and SSLUtil.setEnabledSSLCipherSuites methods, and it looks like you’re using those in your code.

It’s definitely weird that your output shows that the later invocation of TLSCipherSuiteSelector only has TLSv1.2 and TLSv1.3 enabled. That comes directly from SSLUtil.getEnabledSSLProtocols, so I have no idea why that output would not be aligned.

There are only a couple of things that I can think of that might potentially be responsible for this:

• In the code that you use to set the supported cipher suites, it looks like the only suite that you list that is applicable to TLSv1 and TLSv1.1 is “TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA”. I believe that all of the others listed are applicable to TLSv1.2. Is it possible that the server doesn’t support the TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA cipher suite, and therefore you are ending up with no cipher suites in common for the negotiated protocol version?

• On a related note, I can see that the set of cipher suites that you are providing includes some duplicates. In particular, the TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA suite appears four times, and both TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 and TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 appear five times. Is it possible that somehow Scala’s set logic is causing the duplicates to end up omitted from the resulting set?

• The LDAP SDK uses a static initializer in SSLUtil (which calls TLSCipherSuiteSelector) to set the initial values for TLS protocol versions and cipher suites. I’m not familiar with the inner workings of Scala, but is it possible that it does static initialization differently than Java, and that somehow your calls to setEnabledSSLProtocols and setEnabledSSLCipherSuites are being negated by a subsequent initialization? Perhaps you could try creating a dummy SSLUtil instance before invoking those methods to ensure that all of the static initialization is out of the way first before you try to override the defaults?

If none of the above help, then I would recommend trying a couple of other things:

• Rather than calling the SSLUtil.setEnabledSSLProtocols and SSLUtil.setEnabledSSLCipherSuites methods, maybe you could try specifying them by having the com.unboundid.util.SSLUtil.enabledSSLProtocols and com.unboundid.util.SSLUtil.enabledSSLCipherSuites system properties set when launching the JVM. If those properties are set, then they override what the LDAP SDK would otherwise set by default, so even if there is some weirdness around the static initializer, I expect that should work around it.

• Try writing a simple Java program that tries to reproduce the problem. If you can reproduce it with Scala but not with Java, then that definitely points to the issue having something to do with the difference between the two languages and the bytecode that gets generated for each. If you can reproduce it with both Scala and Java, then if you could provide me with the Java program, then maybe I’ll be able to reproduce the problem and track it down more easily. Either way, it should help get to the root of the problem more quickly.

[GeluOltean]

Sorry for the delay. I will try out the suggestions and report back with the code examples by the end of tomorrow at the latest. Thank you for the detailed response!

[GeluOltean]

It turns out we had to manually enable TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA. I've added two runnable files, one Scala and one Java, with the .txt extension to be able to upload to Github.

Not sure if this is an issue, however, the JVM debug options will list only TLSv1.2 and TLSv1.3 for both the Java and Scala versions in all the following scenarios:

• Static-like execution via Scala Object type
• Enabling in the main method in Java
• Static block execution in Java

Providing them as system properties to the JVM does work as expected, and we'll be using this option going forward.

Thank you for the support!

LdapTestScala.txt LdapTestScala.log LdapTestJava.txt LdapTestJava.log

[mircea-cm]

Hi @GeluOltean We had the same problem, and by using your approach it works.

private void addTLS1Protocols() { String[] missingCiphers = new String[]{ "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256", "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA", "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256", "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256", "TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA", "TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384", "TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA", "TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA", "TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384", "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256", "TLS_DHE_DSS_WITH_AES_256_GCM_SHA384", "TLS_DHE_RSA_WITH_AES_256_GCM_SHA384", "TLS_DHE_DSS_WITH_AES_128_GCM_SHA256", "TLS_DHE_DSS_WITH_AES_256_CBC_SHA", "TLS_DHE_RSA_WITH_AES_256_CBC_SHA", "TLS_DHE_DSS_WITH_AES_128_CBC_SHA", "TLS_DHE_RSA_WITH_AES_128_CBC_SHA", "TLS_RSA_WITH_AES_256_GCM_SHA384", "TLS_RSA_WITH_AES_128_GCM_SHA256", "TLS_RSA_WITH_AES_256_CBC_SHA256", "TLS_RSA_WITH_AES_256_CBC_SHA", "TLS_RSA_WITH_AES_128_CBC_SHA256", "TLS_RSA_WITH_AES_128_CBC_SHA" }; Set ciphers = new HashSet(); ciphers.addAll(SSLUtil.getEnabledSSLCipherSuites()); ciphers.addAll(Arrays.asList(missingCiphers)); SSLUtil.setEnabledSSLCipherSuites(ciphers); SSLUtil.setEnabledSSLProtocols(Arrays.asList(SSLUtil.SSL_PROTOCOL_SSL_3, SSLUtil.SSL_PROTOCOL_TLS_1, SSLUtil.SSL_PROTOCOL_TLS_1_1, SSLUtil.SSL_PROTOCOL_TLS_1_2, SSLUtil.SSL_PROTOCOL_TLS_1_3)); }

and then addTLS1Protocols(); SSLUtil sslUtil = new SSLUtil(new TrustAllTrustManager()); SSLSocketFactory socketFactory = sslUtil.createSSLSocketFactory(SSLUtil.SSL_PROTOCOL_TLS_1_1); template = new LDAPConnection(socketFactory, this.serverAddress, this.serverPort);

It worked out of the box, without the need to enable them via system properties.

EDIT: code markup doesn't seem to work

[GeluOltean]

This might be something related to how the .jar files are generated by SBT. Anyway, since Mircea mentioned the Java approach works well, and on our side the system property resolves the problem, I think the issue can be closed. Thank you for the support, Neil!