pulsar-helm-chart ssl3 ext invalid servername

Describe the bug Hello, I'm using your helm charts but getting stuck trying to use the Python pulsar-client library from outside the cluster.

To Reproduce 1 - Deploy the helm charts in a Kubernetes cluster. 2 - Enable ingress for pulsar-proxy. 3 - Use the code below:

import pulsar
client = pulsar.Client('https://pulsar-proxy.domain:443')
producer = client.create_producer('my-topic')

Expected behavior Create a connection and a topic called 'my-topic'.

Error ssl3 ext invalid servername (SSL routines, ssl3_ctrl): Error while setting TLS SNI

Additional context It looks like this variable in the reply (proxyThroughServiceUrl) should be 1 for it to use the service URL as proxy and not try to go direct to the cluster using pulsar protocol on port 6650, as that is not available from outside the cluster.

I tried using the additional listeners feature without luck.

Logs

2024-09-18 09:17:57.471 INFO  [139927618582208] HTTPLookupService:223 | Curl Lookup Request sent for https://pulsar-proxy.domain:443/admin/v2/persistent/public/default/my-topic/partitions?checkAllowAutoCreation=true
2024-09-18 09:17:57.642 INFO  [139927618582208] HTTPLookupService:239 | Response received for url https://pulsar-proxy.domain:443/admin/v2/persistent/public/default/my-topic/partitions?checkAllowAutoCreation=true responseCode 200
2024-09-18 09:17:57.642 INFO  [139927618582208] HTTPLookupService:280 | parsePartitionData = { LookupDataResult [brokerUrl_ = ] [brokerUrlTls_ = ] [partitions = 0] [authoritative = 0] [redirect = 0] proxyThroughServiceUrl = 0] }
2024-09-18 09:17:57.643 INFO  [139927618582208] HandlerBase:111 | [persistent://public/default/my-topic, ] Getting connection from pool
2024-09-18 09:17:57.643 INFO  [139927618582208] HTTPLookupService:223 | Curl Lookup Request sent for https://pulsar-proxy.domain:443/lookup/v2/topic/persistent/public/default/my-topic
2024-09-18 09:17:57.738 INFO  [139927618582208] HTTPLookupService:239 | Response received for url https://pulsar-proxy.domain:443/lookup/v2/topic/persistent/public/default/my-topic responseCode 200
2024-09-18 09:17:57.739 INFO  [139927618582208] HTTPLookupService:315 | parseLookupData = { LookupDataResult [brokerUrl_ = pulsar://pulsar-broker:6650] [brokerUrlTls_ = ] [partitions = 0] [authoritative = 0] [redirect = 0] proxyThroughServiceUrl = 0] }
2024-09-18 09:17:57.739 INFO  [139927618582208] ClientConnection:187 | [<none> -> ] Create ClientConnection, timeout=10000
2024-09-18 09:17:57.741 ERROR [139927618582208] ClientConnection:268 | ssl3 ext invalid servername (SSL routines, ssl3_ctrl): Error while setting TLS SNI
2024-09-18 09:17:57.741 INFO  [139927618582208] ConnectionPool:124 | Created connection for --0
2024-09-18 09:17:57.741 ERROR [139927618582208] ClientConnection:579 | [<none> -> ] Invalid Url, unable to parse: system:0 Success
2024-09-18 09:17:57.741 ERROR [139927618582208] ClientConnection:1318 | [<none> -> ] Connection closed with ConnectError (refCnt: 2)
2024-09-18 09:17:57.741 INFO  [139927618582208] ConnectionPool:141 | Remove connection for --0
2024-09-18 09:17:57.742 INFO  [139927618582208] ClientConnection:275 | [<none> -> ] Destroyed connection to -0
2024-09-18 09:17:57.742 INFO  [139927618582208] ProducerImpl:757 | Producer - [persistent://public/default/my-topic, ] , [batching  = off]

Sep 18 '24 08:09 meyerbro

client = pulsar.Client('https://pulsar-proxy.domain:443')

Does it work if you use the Pulsar binary protocol (pulsar+ssl://pulsar-proxy.domain:6651)?

Sep 19 '24 04:09 lhotari

client = pulsar.Client('https://pulsar-proxy.domain:443')

Does it work if you use the Pulsar binary protocol (pulsar+ssl://pulsar-proxy.domain:6651)?

Hello, unfortunately not as the proxy doesn't expose 6651 but 443 (ingress).

Sep 25 '24 13:09 meyerbro

Hello, unfortunately not as the proxy doesn't expose 6651 but 443 (ingress).

@meyerbro Do you have 443 mapped to Pulsar binary protocol? You cannot proxy Pulsar binary protocol over HTTPS. Pulsar client can use https for lookups, but it needs Pulsar binary protocol to connect to a broker. (alternative is SNI proxy, but that's not covered in the Pulsar Helm chart) In the default values.yaml, pulsarssl port is 6651.

  ports:
    http: 80
    https: 443
    pulsar: 6650
    pulsarssl: 6651

Sep 25 '24 14:09 lhotari

Hello, unfortunately not as the proxy doesn't expose 6651 but 443 (ingress).

@meyerbro Do you have 443 mapped to Pulsar binary protocol? You cannot proxy Pulsar binary protocol over HTTPS. Pulsar client can use https for lookups, but it needs Pulsar binary protocol to connect to a broker. (alternative is SNI proxy, but that's not covered in the Pulsar Helm chart) In the default values.yaml, pulsarssl port is 6651.
  ports:
    http: 80
    https: 443
    pulsar: 6650
    pulsarssl: 6651

I created an ingress that points to 6650 of the pulsar-broker pod (I only have 1 replica) and then I just try to hit that from a python client on my windows machine (outside the kube cluster).

Can you disclose more? I'm really happy with the support I'm getting here, thank you.

Sep 25 '24 14:09 meyerbro

We use nginx ingress, wondering if I can do the SNI config on that.

Sep 25 '24 15:09 meyerbro

Can you disclose more? I'm really happy with the support I'm getting here, thank you.

#423 contains some related information.

We use nginx ingress, wondering if I can do the SNI config on that.

it should be possible, but you are on your own.

Sep 25 '24 15:09 lhotari