Hostname verification should include IP Address SAN verification (RFC2818 requirement)

[Ralfers]

Description

The librdkafka configuration properties list has the following description for the "ssl.endpoint.identification.algorithm" property:

"Endpoint identification algorithm to validate broker hostname using broker certificate. https - Server (broker) hostname verification as specified in RFC2818."

RFC2818 also has the following requirement:

"In some cases, the URI is specified as an IP address rather than a hostname. In this case, the iPAddress subjectAltName must be present in the certificate and must exactly match the IP in the URI."

The librdkafka library does not check the IP Address SAN so it does not conform to the RFC2818 fully.

At the very least, would like to see a note on this limitation, maybe next to the"ssl.endpoint.identification.algorithm" property.

Thanks!

How to reproduce

1. Set up a kafka broker with SSL and a client certificate, containing the IP Address SAN;
2. Set the kafka broker "advertised.listeners" property to "SSL://<ip>:9093";
3. Set up librdkafka with SSL and hostname verification;
4. Set the librdkafka property "bootstrap.servers" to "<ip>:9093";
5. try to produce a message to some topic in the broker.

Logs

From librdkafka: "SSL handshake failed: s3_clnt.c:1264: error:14090086:SSL routines:ssl3_get_server_certificate:certificate verify failed: broker certificate could not be verified, verify that ssl.ca.location is correctly configured or root CA certificates are installed (install ca-certificates package) (after 14ms in state CONNECT) (_SSL): identical to last error: error log suppressed"

From the kafka broker:

Checklist

Please provide the following information:

• [x] librdkafka version: v1.6.2
• [x] Apache Kafka version: 3.1.0
• [x] librdkafka client configuration: bootstrap.servers=<broker_ip>:9093,broker.address.family=v4,enable.idempotence=true,message.timeout.ms=10000,security.protocol=SSL,ssl.endpoint.identification.algorithm=https,ssl.keystore.location=<keystore_path>,ssl.keystore.password=***,ssl.key.password=***,ssl.ca.location=<ca_path>
• [x] Operating system: Oracle Linux Server 7.9
• [x] Provide logs (with debug=.. as necessary) from librdkafka
• [x] Provide broker log excerpts
• [ ] Critical issue

[Orrimp]

We have the same problem. Kafka over Spring Boot works but not through kcat using identical files and values for broker configuration.

[P1ng-W1n]

With the 2.0 release (and host verification enabled by default) the visibility of this issue has increased too. Connectivity with openssl / Kafka clients (like kafka-topics) do work with provided certificates / settings, but fails to be established with librdkafka based clients. In CI / lab environments (where IP based connections are more frequent) this is causing additional headaches.

[LaithS]

The issue does not appear if you link against OpenSSL-3, the implementation of SSL_set1_host is changed in openSSL3 to attempt to parse hostname as an IP address and if so enable checking against ipAddress SANs. https://www.openssl.org/docs/man1.1.1/man3/SSL_set1_host.html

[jcea]

Being hit by this too. Any progress/schedule?

[riccardomanfrin]

Same here.. I used to use BIO_set_conn_hostname which

"uses the string name to set the hostname. The hostname can be an IP address;".

I replaced with SSL_set1_host but this one

sets the expected DNS hostname

I suppose this one doesn't like ip adresses :(