TLS Handshake record layer fragmentation not working

[Irrialite]

I have a server setup that sends handshake with ssl->in_hslen set to above 22700-ish, which is above the accepted limit of 16384. Is there any plan to support this (or a workaround without changing the server) and if not, I would appreciate some tips as to how to implement this as it is quite urgent for my project.

This is the code segment that fails: /* With TLS we don't handle fragmentation (for now) */ if( ssl->in_msglen < ssl->in_hslen ) { MBEDTLS_SSL_DEBUG_MSG( 1, ( "TLS handshake fragmentation not supported" ) ); return( MBEDTLS_ERR_SSL_FEATURE_UNAVAILABLE ); }

[RonEld]

@Irrialite As you can see from this PR we are working an refactoring the messaging infrastructure. Message fragmentation being one of the issues addressed. Unfortunately, as you can see, this is still work in progress, and we cannot commit on when it will be ready.

[RonEld]

You are welcome to try this PR and ad your comments though.

[RonEld]

Note that this PR does require integration with the rest of the library though.

[RonEld]

@Irrialite I have just looked again at your query. Please note that TLS specification allows only up to 16 KB for the record data, so even once we support message fragmentation, sending 22700-ish is not allowed. The application should only send 16 KB of data. If the data is larger, then perhaps the application should send it in several chunks, and the remote peer application should be able to reconstruct the whole data buffer.

[RonEld]

@Irrialite I would like to clarify, that my previous comment was about the Application data. I now realize you were asking about the Handshake message length. The standard doesn't limit the length of the handshake message( except that the length field is 24 bits, so the message cannot exceed 32MB).

In addition, the Fragmentation support which is being worked on, is about the DLTS fragmentation. Supporting TLS fragmentation is not at the moment planned, as usually it's not a problem. Usually, the bottle neck of the handshake messages are the certificate messages. Is it possible that your server would send a shorter certificate chain? Have you considered using ECC based certificates?

[RonEld]

We believe this issue has been addressed, therefore closing. If you believe this issue remains unclear or unanswered, please reopen, with an update

[RonEld]

Reopening, for the sake of tracking the TLS handshake fragmentation feature

[rojer]

@RonEld this is more serious when Max Fragment Length is used. if a device (using mbedTLS) asks for a small fragment, e.g. 1024 bytes, and the server complies, then handshake ends up fragmented:

handshake message: msglen = 1024, type = 11, hslen = 2675

which leads to the "fragmented handshake" failure.

[RonEld]

@rojer Thank you for your example. You example is exactly using the max fragment length. (MFL) MFL is cucrrently supported for DTLS only. I suggest you look at this article for more information. In TLS, however, since the bottle neck is probably the certificate message, it is recommended you set your MBEDTLS_SSL_IN_CONTENT_LEN value to fit the certificate size. in your case, probably bigger than 2675.

[bl4ck5un]

@RonEld I'm running into the same problem while working on a client. So mbedtls_ssl_conf_max_frag_len is not supported for client either? I tried to set MBEDTLS_SSL_IN_CONTENT_LEN to 5120 without calling mbedtls_ssl_conf_max_frag_len. But the server still sends me messages longer than that.

[bl4ck5un]

Reading the code, I called mbedtls_ssl_conf_max_frag_len(&conf, MBEDTLS_SSL_MAX_FRAG_LEN_NONE) as well, thinking that's going to cause the client to negotiate with the server, which apparently didn't happen. Am I missing anything?

[oesh]

We (Facebook networking team) have started working on adding handshake fragmentation support to mbedTLS. We are going to upstream the patch to the master repository once it is ready.

[rojer]

that would be great! looking forward to it. would be super nice to also have it backported to 2.16

[bl4ck5un]

@oesh that will be wonderful!! I've been blocked by this for more than a year. How is your implementation going? Do you have a public dev branch that I might try?

[oesh]

@bl4ck5un hey, I was busy with something else, but I am back to it. I do have a public dev branch, and it will be terrific to see early reports from the users. I will post an update here once I have something that passes tests, probably in the course of the following two weeks.

[oesh]

that would be great! looking forward to it. would be super nice to also have it backported to 2.16

@rojer I am starting with 2.16.6, since this is what we need internally.

[oesh]

Sharing some of the design considerations. Please comment as appropriate.

Memory management

MbedTLS expects the handshake message to be in ssl->in_buf. ssl->in_buf contains MBEDTLS_SSL_IN_BUFFER_LEN, which is sufficient for a single TLS record. The fragmented handshake message will not necessarily fit in MBEDTLS_SSL_BUF_LENGTH.

I see several design alternatives, from the simplest to the more complicated. All assume a new config constant, MBEDTLS_SSL_HS_MSG_MAX_LEN, which limits the size of a single handshake message.

Simplest

The simpler alternative is to increase the MBEDTLS_SSL_IN_BUFFER_LEN to be sufficient for the individual handshake message.

Advantages

1. No changes to the TLS state machine. Easier to reason about.

Disadvantages

1. The runtime memory footprint of a session will be permanently increased, by MBEDTLS_SSL_HS_MSG_MAX_LEN - MBEDTLS_SSL_MAX_CONTENT_LEN, which is likely to be 64K - 16K == 48K.

Agressive memory footprint optimization

Instead of passing ssl->in_msg to the TLS state machine, use a different pointer, e.g. ssl->in_hs_msg, which can point to either the ssl->in_msg, if the handshake message was not fragmented, or to a different buffer, e.g. ssl->handshake->hs_msg (similar to the DTLS handshake reassembly code).

Advantages

1. The memory footprint will likely to be minimal.

Disadvantages

1. This is by far the most complex approach, and it will be harder to debug and maintain.

KISS (for now)

Considering the above alternatives, I am opting for the simpler alternative for the time being. My reasoning is:

1. The reassembly of the handshake messages can be disabled in the config parameters, hence the users who do not need the reassembly will suffer no runtime penalty.
2. The simplest alternative is by far the easiest to get right, and will result in a smaller diff.
3. It will be still possible to add memory optimizations later, in an incremental fashion.

Edit: adjusted the design alternatives to the 2.16 branch, which makes the "slightly optimized" alternative the simplest.

[Vitaliy69]

Any progress?

Call on my IoT STM32 device:

mbedtls_ssl_conf_max_frag_len(&conf, MBEDTLS_SSL_MAX_FRAG_LEN_512);

and got the error:

Performing the SSL/TLS handshake.../build/mbedtls-mygkMM/mbedtls-2.16.4/library/ssl_tls.c:3698: 0x7ffd936d4c80: TLS handshake fragmentation not supported
/build/mbedtls-mygkMM/mbedtls-2.16.4/library/ssl_tls.c:4369: 0x7ffd936d4c80: mbedtls_ssl_handle_message_type() returned -28800 (-0x7080)

[oesh]

Update:

Support for fragmented TLS handshake messages is ready for a merge into the development branch. At the moment I am waiting for the final sign-off by the MbedTLS core devs.

In the meantime, the implementation is available in https://github.com/oesh/mbedtls/tree/hs_fragmentation__design_doc

If you want to try this out:

1. Ensure that MBEDTLS_SSL_HS_DEFRAG_CLT is enabled in include/mbedtls/config.h
2. If needed, adjust the maximal size of an individual handshake message via MBEDTLS_SSL_HS_DEFRAG_MAX_MSG_LENGTH. The default maximal size is 16384.
3. Enable defragmentation in the runtime by invoking mbedtls_ssl_conf_hs_defrag_max_len( conf, len ) when setting up the TLS context. The len parameter is the maximal size of a handshake message, which should not exceed the MBEDLTS_SSL_HS_DEFRAG_MAX_MSG_LENGTH configuration parameter.

[oesh]

that would be great! looking forward to it. would be super nice to also have it backported to 2.16

@rojer I am starting with 2.16.6, since this is what we need internally.

Update - I have asked the MbedTLS community whether there's any issue with backporting the feature to 2.16. The response was that since this is not a bug fix, it would be best to not risk the stability of the long-term-support versions:

https://lists.trustedfirmware.org/pipermail/mbed-tls/2020-October/000205.html

[rojer]

fwiw, Cesanta agreed to open source our mbedtls modifications, among them i have incoming handshake message defragmentation - https://github.com/mongoose-os/mbedtls/commit/097ccf2ff88f30e894cd14c80409498bbb02fa10

this is against 2.16, and to complete the work fragmentation of outgoing messages is required. i do plan to finish this work at some point, but no firm ETA.

[lance-proxy]

Hi all, any updates on this issue above? It seems like message fragmentation is becoming more and more common and going to be problem for IoT devices to not have support for this.

[rnhmjoj]

It's particularly annoying because mbedtls seems to make some webserver return a fragmented handshake, even if well below the 16K limit. This is one such handshake (should be easily reproduced by visiting any website using netlify.com):

Do you know what is going on? I tried comparing mbedtls and openssl client hellos but nothing stands out:

In particular, it's not asking a max_fragment_length, which has been mentioned above.

[hanno-becker]

@rnhmjoj Nothing stands out to me either - note that it's every implementation's choice to pick a maximum record plaintext size.

We have fragmentation working in our experimental TLS 1.3 implementation and are in the process of upstreaming it. cc @daverodgman

[lance-proxy]

@hanno-arm would love to give it a try on my device, could you point me to the branch for the experimental TLS 1.3 implementation ?

[hanno-becker]

@lance-proxy The branch is currently still on @hannestschofenig's fork https://github.com/hannestschofenig/mbedtls/tree/tls13-prototype/, but we'll move it to the main repository soon. Please let me know if you have any problems - the quickest way is probably to open an issue on the fork - any feedback is welcome.

[ashesman]

Has there been any further movement on this? It seems to me rare that certificates are small enough to be sent without fragmentation, especially if smaller fragmentation sizes are used. At the moment you can not take advantage of fragmentation for reduced memory foot print unless you can be sure that certificates will be small. TLS is required to support fragmentation starting from handshake so seems like it would be a good thing to have.

[rojer]

I've finished work mentioned in https://github.com/Mbed-TLS/mbedtls/issues/1840#issuecomment-753009225 and we've had these changes in production for more than a year. As part of rebasing to 3.x i've now forward-ported the changes to 3.x and put up https://github.com/Mbed-TLS/mbedtls/pull/8981. Feedback/test reports would be appreciated.

[zliucd]

This may often happen when TLS server sends certificates in different and sequential packets. We can use len field from TLS record header to identify missing bytes and parse them.

Though I have solved this issue in my personal project(https://github.com/zliucd/TLSfeat), but why fragmentation happens is still unknown.

[rojer]

why fragmentation happens is still unknown

wdym unknown? handshake fragmentation happens because it is being requested by the client.