libwebsockets icon indicating copy to clipboard operation
libwebsockets copied to clipboard

lws/openssl-wrapper: Align ssl_handshake and openssl standard return values

Open ThePassionate opened this issue 4 months ago • 1 comments

https://docs.openssl.org/1.0.2/man3/SSL_do_handshake/#return-values standard return code of ssl_handshake list following: 0: The TLS/SSL handshake was not successful but was shut down controlled and by the specifications of the TLS/SSL protocol. Call SSL_get_error() with the return value ret to find out the reason.

1: The TLS/SSL handshake was successfully completed, a TLS/SSL connection has been established.

<0: The TLS/SSL handshake was not successful because a fatal error occurred either at the protocol level or a connection failure occurred. The shutdown was not clean. It can also occur of action is need to continue the operation for non-blocking BIOs. Call SSL_get_error() with the return value ret to find out the reason.

so ssl_error_read/write should return -1. Also mapping ssl->err from mbedtls error to openssl error, not return mbedtls error directly.

ThePassionate avatar Aug 14 '25 03:08 ThePassionate

I notice that we change the return code meaning, but we don't adapt the caller. Since mbedtls works today, are we sure that is a good idea?

lws-team avatar Aug 15 '25 06:08 lws-team