SSL Session reuse with SSL Passthrough and wildcard certificate

[primeroz]

This is not a Bug with the ingress or haproxy itself but is an edge case i ended up hitting and have been wondering if there is anything we can do on the ingress controller side to address it or in alternative something to document on the docs

see https://discourse.haproxy.org/t/sni-routing-to-servers-only-ever-sent-to-first-server/1265

scenario

When using a wildcard "*.example.com" certificate as default certificate for a number of ingresses as well as one ssl passthrough the client reuse of ssl session to avoid re-performing the ssl handshake can cause wrong routing , to the default backend probably, to happen. Server Name information is sent with each SSL Handshake, whether you’re establishing a new session or you’re resuming an old one

assume you have the following ingress

• foo.example.com - using the wilcard cert , routed based on host header
• bar.example.com - using the wilcard cert , routed based on host header
• passthrough.example.com - using ssl passthrough , routed based on sni

reproduce by

• Open an incognito window to start with a clear ssl cache

• open a tab to https://passthrough.example.com

• You will hit the passthrough service

• Open an incognito window to start with a clear ssl cache

• open a tab to https://foo.example.com

• open a tab to https://passthrough.example.com

• You will hit the default backend

Why

Using a 0.7.5 Configuration example ( but i don't see why it would not happen on 0.8 even though i did not test it yet and about to do it )

frontend httpsfront                                                                                                                                    
    bind *:443                                                                                                                                         
    mode tcp                                                                                                                                           
    tcp-request inspect-delay 5s                                                                                                                       
    tcp-request content accept if { req.ssl_hello_type 1 }                                                                                             
    use_backend passthrough-backend-8443 if { req.ssl_sni -i passthrough.example.com }                                                          
    default_backend httpsback-shared-backend  

When the ssl handshake is performed for passthrough.example.com the SNI routing works and the right backend is used and everything works

When the ssl handshake is performed with foo.example.com when a new tab to passthrough.example.com is opened the acl for sni fails , the request is then sent to the default_backend which has no match for the host header and so the request end up using the default backend.

What i expect from this issue ?

Hopefully any idea on how to fix this issue without having to separate ingresses based on those that use the wildcard certificate (99% of my ingresses) vs the ones using ssl passthrough

[jcmoraisjr]

Hi, v0.9+ addresses this issue on backends secured by tls certificates due to security reasons - a 421 http response is sent to the client whenever sni and host header doesn't match. This same approach can be used on ssl-passthrough as well, but it would fix the behaviour only if the non passthrough url is issued first. If you first issue the ssl-passthrough url, haproxy won't have a way to see if the sni and host header match because the ssl-passthrough domain would be called and the header is inside the encrypted data.

In the mean time you can try the following:

1. Use distinct certificates. This will fix the problem but I'm aware it's not that trivial
2. Change alpn to http/1.1 - if you are lucky the tls connection won't be reused if a h2 connection cannot be stablished. This would however break any h2 backend like grpc.

[primeroz]

I understand.

Yeah i was thinking to ditch the wildcard certificate but after thinking about it it was going to be way too much effort since i really only have 1 ingress with this issue out of the tens i run

I ended up for now just using a tcp service mapping so i get a dedicated pool on a different port and in my case it was good enough since this special ingress is a service rather than a website

Maybe is just something we should document on the docs so that people are aware of it ? It so does not feel like something the ingress side can reliably address ( see the if you issue the ssl-passthrough first)

[bars83]

You can use balancing by host name, not by SNI

After TLS connection with SNI, browser reuse it, and HAProxy always see SNI as in first request. Even if next request have another Host header.

I got it work after change

use_backend my-backend-1 if { ssl_fc_sni my1.example.com }
use_backend my-backend-2 if { ssl_fc_sni my2.example.com }

to

acl my_host_1 hdr(host) -i my1.example.com
acl my_host_2 hdr(host) -i my2.example.com
use_backend my-backend-1 if my_host_1
use_backend my-backend-2 if my_host_2