OCSP stapling interop issues

[voltone]

Describe the bug During testing of the new OCSP stapling client feature, interop issues with various servers were observed. I would be happy to create separate tickets for each, if that's preferred, but I thought at first it might be easier to review them together and see which ones are considered bugs.

1. When OCSP stapling is requested, ssl defaults to sending a status_request with a nonce; this is highly unusual and many servers are not prepared to handle this, as it would require for the HTTP server to reach out to the OCSP server to obtain a fresh OCSP response rather than stapling a cached reponse

2. When OCSP stapling without nonce is requested, ssl still sends extensions in the status_request; this is unusual and seems unnecessary

3. When a server does not support OCSP stapling, ignores the status_request extension, does not echo it in the Server Hello and does not send a Certificate Status handshake message, ssl aborts the TLS handshake; this would be appropriate if the server certificate had a "must-staple" extension, but otherwise some sort of fallback would be desirable

4. With some servers the TLS handshake fails with and internal error: TLS client: In state certify at ssl_handshake.erl:361 generated CLIENT ALERT: Fatal - Internal Error\n {unexpected_error,{bad_generator,asn1_NOVALUE}}

To Reproduce

Preparation for all of these: make sure the inets and ssl applications have been started.

1. Apple and CNN both ignore the nonce and staple a cached OCSP response:

• httpc:request(get, {"https://www.apple.com", []}, [{ssl, [{cacerts, public_key:cacerts_get()}, {stapling, staple}]}], []).
• httpc:request(get, {"https://www.cnn.com", []}, [{ssl, [{cacerts, public_key:cacerts_get()}, {stapling, staple}, {customize_hostname_check, [{match_fun, public_key:pkix_verify_hostname_match_fun(https)}]}]}], []).
• Response in both cases: {bad_certificate,"TLS client: In state wait_cert_cr at ssl_handshake.erl:2106 generated CLIENT ALERT: Fatal - Bad Certificate\n nonce_mismatch"}

2. Can be observed by making a tcpdump/Wireshark trace while running e.g.:

• httpc:request(get, {"https://www.apple.com", []}, [{ssl, [{cacerts, public_key:cacerts_get()}, {stapling, #{ocsp_nonce => false}}]}], []).
• 

3. Example with Erlang website:

• httpc:request(get, {"https://erlang.org", []}, [{ssl, [{cacerts, public_key:cacerts_get()}, {stapling, staple}]}], []).
• Response: {bad_certificate,"TLS client: In state certify at ssl_handshake.erl:2106 generated CLIENT ALERT: Fatal - Bad Certificate\n not_stapled"}

4. Examples with IETF and DuckDuckGo:

• httpc:request(get, {"https://ietf.org", []}, [{ssl, [{cacerts, public_key:cacerts_get()}, {stapling, #{ocsp_nonce => false}}]}], []).
• httpc:request(get, {"https://duckduckgo.com/", []}, [{ssl, [{cacerts, public_key:cacerts_get()}, {stapling, #{ocsp_nonce => false}}]}], []).
• Response in both cases: {tls_alert,{internal_error,"TLS client: In state wait_cert_cr at ssl_handshake.erl:361 generated CLIENT ALERT: Fatal - Internal Error\n {unexpected_error,{bad_generator,asn1_NOVALUE}}"}

Expected behavior

1. None of the browsers I have tested send a nonce in the status_request, and not many servers appear to support it; I would recommend making nonce opt-in, so default to #{ocsp_nonce => false}

2. The browsers I have tested send a minimal status_request in the Client Hello; if no nonce is present I think it might be best to do the same, to reduce overhead and maximize interoperability

• 

3. It seems the behavior can be customized using a verify_fun that handles the{bad_cert,{revocation_status_undetermined,not_stapled}} Reason value. But that's a lot of hoops to jump through to achieve the behavior that's common in other clients: try to ask the server if they would be so kind as to staple an OCSP response, and if they don't, fall back on some other revocation check

4. I would expect the connection to succeed, unless the server's response is actually illegal (I have not yet gotten to the bottom of this, but other TLS clients do not appear to have a problem with it)

Affected versions 27.0-rc1

[IngelaAndin]

Thank you for the report we will look into it for the 27 release. No need to make several issues at the moment.

[u3s]

thanks a lot for detailed issue report. we will investigate it and get back soon.

[u3s]

• items 2, 4 are fixed in #8353 (note: there seems to be one more issue with ietf.org certificate, but it might be not OCSP related at this stage - investigation is ongoing)
• items 1,3 - are kind of design choices - security vs interoperability. we have not finalized internal discussion about it yet, but we are leaning towards having more secure behavior in ssl being a default and maybe being able to customize it in certain contexts. maybe inets client could favor interoperability more similar to web browsers ... @IngelaAndin what do you think?

[voltone]

Thanks for the fixes! My 2c about the others:

It is great to see the move towards "secure by default" in ssl in recent OTP releases, but when it comes to "security versus usability" I would suggest to also consider AviD's Rule of Usability:

Security at the expense of usability, comes at the expense of security.

In this case, if we make enabling revocation checks hard to work with in practice, then people will just disable revocation checks altogether. I personally think a fresh, cached OCSP response without a nonce should provide a good indication that the certificate is still valid and hasn't been revoked. It reduces the time window in which a revoked certificate may be (ab)used to a few hours/days, instead of months. But indeed, that is a design choice.

About (3), there currently isn't really much to fall back on: ideally there would be an OCSP client that can try to get an OCSP response directly from the CA when the server doesn't staple. If the client knows in advance that the legitimate server supports stapling, then the absence of a stapled response would be suspicious, and might indicate an attacker is trying to use a compromised (and revoked) certificate in a MitM attack. In practice the client usually doesn't know that in advance (unless the certificate has a "must staple" extension), and OCSP stapling is more of an optimization to try and save the client the round-trip to the OCSP server. But only if there is an OCSP client to fall back on...

[IngelaAndin]

Well revisiting the OCSP specs the nonce seems to a feature of the original OSCP protocol to prevent replay attacks. So maybe we do not need it for the stapling? If I understand it correctly it may more be an option for the server to use when it acts as a OSCP client, or if we should fallback later on normal OCSP, we where actually talking about falling back on CRL checks if they where configured. As I understand it stapling will increase security over normal OCSP and therefor we where thinking of not supporting normal OSCP but only stapling.

[IngelaAndin]

@u3s what is it that cause the bad_generator error? I do not understand that one at the moment!

Ok, we talked about this in a meeting, and think this is handled appropriately now.

[IngelaAndin]

I guess the nonce we have configures what the server does, that hopefully has a securer connection to the server then the client would and maybe we can live with out having it on by default !?

[u3s]

I've read the RFCs, which confirmed that implementation following specs should behave as you describe (items 1 and 3). For 1-3, I hope to merge a change to master this week. https://github.com/erlang/otp/pull/8431

@voltone - I think this issue report helped us a lot in fixing code before release. I appreciate this and would like to thank you in stapling release note. Maybe in a form of: "Thanks to voltone for interop testing and related discussions." or something like that. Of course if you do not wish it - please let me know.

[voltone]

@u3s glad I could help: contributing OCSP support to OTP has long been on my list of things I wanted to do, but I never got around to it. Nice to know that the feedback was helpful.

[u3s]

With #8431 client should behave as we agreed. Planned to be included in upcoming release.

[u3s]

closing issue as related PR are merged to master. @voltone thanks a lot for contribution!

Release note is planned to look like this: "The ssl client can negotiate and handle certificate status request (OCSP stapling support on the client side). Thanks to voltone for interop testing and related discussions."

If you would like to adjust something - let me know.