Add health probe protocol annotation

[MartinAbadie]

Hi,

Issue

I would like expose my app on port 8443 with HTTPS protocol for public enpoints, and expose on port 8080 with HTTP protocol only for health probes endpoints. Unfortunatly, if I specify appgw.ingress.kubernetes.io/backend-protocol: "https" and appgw.ingress.kubernetes.io/health-probe-port: "8080" annotations, created health probes rule uses HTTPS on port 8080.

Solution

It could be usefull to be able to specify health probe protocol with this kind annotation : appgw.ingress.kubernetes.io/health-probe-protocol: "http".

Attachments

Liveness and readiness example :

apiVersion: apps/v1
kind: StatefulSet
metadata:
  namespace: my-api
  name: my-api
...
          ports:
            - containerPort: 8443
            - containerPort: 8080
          livenessProbe:
            failureThreshold: 3
            httpGet:
              path: /actuator/health/liveness
              port: 8080
              scheme: HTTP
            initialDelaySeconds: 120
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 5
          readinessProbe:
            failureThreshold: 3
            httpGet:
              path: /actuator/health/readiness
              port: 8080
              scheme: HTTP
            initialDelaySeconds: 45
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 1
...

Ingress manifest example :

# Statefulset extract
kind: Ingress
apiVersion: networking.k8s.io/v1
metadata:
  name: my-api
  namespace: my-api
  annotations:
    kubernetes.io/ingress.class: azure/application-gateway
    appgw.ingress.kubernetes.io/appgw-ssl-certificate: "my-api-tls"
    appgw.ingress.kubernetes.io/appgw-trusted-root-certificate: "my-api-ca"
    appgw.ingress.kubernetes.io/backend-hostname: "my-api.com"
    appgw.ingress.kubernetes.io/backend-path-prefix: "/"
    appgw.ingress.kubernetes.io/backend-protocol: "https"
    appgw.ingress.kubernetes.io/health-probe-port: "8080"
    appgw.ingress.kubernetes.io/health-probe-protocol: "http" # Solution
    appgw.ingress.kubernetes.io/request-timeout: "60"
    appgw.ingress.kubernetes.io/ssl-redirect: "true"
...

Thanks !

Regards

[fatpowaranga]

I believe a workaround at least for the time being if you haven't tried it would be to not specify any health probe via annotations, and instead let it use your readiness probe settings to define the health probe. That is what we are doing currently. (Apologies if I misunderstand)

EDIT: Just to clarify though, I agree this would be a good feature.

[neilkuantm]

any update about this?? I meet the same issue. use caddy server in the backend pool as proxy server.

• caddy deployment

        ports:
        - containerPort: 80 #redirect to https port.
          name: http
          protocol: TCP
        - containerPort: 443 #service port. 
          name: https
          protocol: TCP
        - containerPort: 22
          name: ssh
          protocol: TCP
        - containerPort: 9500 #metrics port
          name: metrics
          protocol: TCP
        readinessProbe:
          failureThreshold: 3
          httpGet:
            path: /ready
            port: 8003 #health check ready port.
            scheme: HTTP
          initialDelaySeconds: 5
          periodSeconds: 5
          successThreshold: 1
          timeoutSeconds: 1

• ingress-config

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: proxy-ingress
  annotations:
    kubernetes.io/ingress.class: azure/application-gateway
    appgw.ingress.kubernetes.io/backend-protocol: "https"
spec:
  tls:
    - hosts:
      - "proxy.com"
      secretName: "https-certificates"
  rules:
  - host: "proxy.com"
    http:
      paths:
      - path: /*
        backend:
          service:
            name: proxy
            port:
              number: 443
        pathType: Prefix

But AGIC help me to created the health probes use HTTPS protocol, it will let backend health check fail...

Because caddy server use HTTP 8003 to check health check...

hi @akshaysngupta ,any idea about this ?! Thank you~~~

updated 2022/04/20

I read docs about application gateway health probes. https://docs.microsoft.com/en-us/azure/application-gateway/application-gateway-probe-overview#default-health-probe

so we can not use appgw.ingress.kubernetes.io/backend-protocol: "https" and appgw.ingress.kubernetes.io/health-probes-protocol: "http"(if support this feature) together ??