Non-blocking DTLS implementation

[timothyjward]

The TLS implementation has been able to work in non-blocking mode for a long time. Java 9+ SSLEngine now supports non-blocking DTLS, and Bouncy Castle is beginning to lag behind.

I did see one OS project which seems to have re-implemented the BC DTLS support in an asynchronous way, but I'm not sure what its provenance is https://github.com/mobius-software-ltd/java-dtls

[peterdettman]

Apologies for the delayed response.

Due to the current code organisation (of DTLS code in particular), non-blocking mode is not at all straightforward unfortunately. I suppose the primary issue is that the DTLS code doesn't share the TLS state machine.

The linked project perhaps shows that it's possible, but also demonstrates the extensive changes needed to rewrite in terms of handlers and events.

[makkarpov]

Actually when you try to do it inside of BouncyCastle, not as separate library, it's really not that bad. Many many parts of existing DTLS stack can be reused, but slightly modified. Mentioned library had to re-invent wheel again and again, since many useful parts are private within BouncyCastle.

I'm almost finished proof-of-concept non-blocking DTLS implementation as part of BouncyCastle library itself, and I will be happy to merge it. However, some guidance will be appreciated. Questions that I see now:

1. Currently non-blocking DTLS implementation is done by copy-pasting existing BouncyCastle code. This is fine for PoC, but unacceptable for real code. So both implementations should share same code base. My suggestion is to make 'sync' implementation wrap async.

2. API that should be exposed to user. Currently I'm thinking of something like:

   interface AsyncDTLSProtocol {
     void pushReceivedData(byte[] data, int off, int len);
     void sendApplicationData(byte[] data, int off, int len);
   }
   interface AsyncDTLSTransport extends DatagramSender {
     void exceptionCaught(Throwable cause);
     void applicationDataReceived(byte[] buffer, int offset, int length);
     void close();
   }
   

   The basic idea is to "push" received data into DTLS engine, and DTLS engine will fire appropriate transport events inside of handler function. This is how state machine runs - you push the incoming data, and engine will either request you to send something back or give you decrypted application data.

   It also involves third interface to handle timeouts, this time very simple:

   public interface TlsTimer {
       interface TimerTask {
           void cancel();
       }
   
       TimerTask schedule(Runnable runnable, long timeoutMillis);
   }
   

   Note that API, despite being async, is intentionally designed as thread-unsafe. This is done because many networking libraries like Netty have threading model that exposes socket handlers as something that does not require synchronization. This is usually done by attaching channels to specific event loops, so single dedicated thread will handle events of that particular channel.

---

Side question: why do we care about 20-year-old forgotten and abandoned JDK's? This rules out many performance improvements, like ByteBuffers in ciphers, and even simple code style. No generics, no enums, no lambdas, for what? J2ME? Is that still alive?

[makkarpov]

https://github.com/makkarpov/async-dtls/commit/c5717a28ba40f0441633f583ffb56b62d51e0cb1

I had finished it. DTLSReliableHandshake is a pure nightmare to debug. Never seen more implicit state machine that breaks in a such subtle ways.

Changes:

• Rather slight modifications to DTLSRecordLayer and DTLSReliableHandshake that introduces pushes instead of receives
• Code of DTLSProtocol, DTLSServerProtocol and DTLSClientProtocol was moved to DTLSAsyncHandshake, DTLSAsyncClientHandshake and DTLSAsyncServer handshake respectively, and large handshake methods were split into state machine.
• DTLS***Protocol classes left as entry-points and factories to handshake methods
• Sync API was re-written as wrapper around async