[helm] add prometheus metrics port to list of ports in the deployment or service file for gateways

[gjelu]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request. Searching for pre-existing feature requests helps us consolidate datapoints for identical requirements into a single place, thank you!
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.

---

Is your feature request related to a problem? Please describe.

We make use of the internal monitoring of Red Hat OpenShift (user workload monitoring to be more precise - https://docs.openshift.com/container-platform/4.10/monitoring/managing-metrics.html#specifying-how-a-service-is-monitored_managing-metrics ).

When enableGatewayMetrics is set to true, the default port 20200 is not added to the list of ports from the container (or in the service definition).

As we need to reference the port name of the container in the PodMonitor CRD, this is not possible today as the port is not listed. The same applies to a ServiceMonitor were we need to reference the port name as defined in the service.

As a dirty workaround, for the ingress gateway, we've added the port 20200 in the list of gateway service ports. However for the termination gateway this is not possible.

        service:
          ports:
            - port: 20200
            - port: 8080

Then we can use something like this to scrape metrics from the ingress gateway:

apiVersion: monitoring.coreos.com/v1
kind: PodMonitor
metadata:
  name: consul-ingress-gateway
  namespace: consul
spec:
  namespaceSelector:
    matchNames:
      - consul
  selector:
    matchLabels:
      app: consul
      component: ingress-gateway
  podMetricsEndpoints:
  - port: gateway-0

Feature Description

Add the port 20200 as metrics port to the list of ports in the container definition.

• https://github.com/hashicorp/consul-k8s/blob/e4776cedd2f941e21e7fc65a36a0730ccdcd0966/charts/consul/templates/mesh-gateway-deployment.yaml#L346
• https://github.com/hashicorp/consul-k8s/blob/e4776cedd2f941e21e7fc65a36a0730ccdcd0966/charts/consul/templates/ingress-gateways-deployment.yaml#L401
• https://github.com/hashicorp/consul-k8s/blob/e4776cedd2f941e21e7fc65a36a0730ccdcd0966/charts/consul/templates/terminating-gateways-deployment.yaml#L356

something like

    {{- if (and .Values.global.metrics.enabled .Values.global.metrics.enableGatewayMetrics) }}
    - name: metrics
      containerPort: 20200
    {{- end }}

This would allow:

apiVersion: monitoring.coreos.com/v1
kind: PodMonitor
metadata:
  name: consul-ingress-gateway
  namespace: consul
spec:
  namespaceSelector:
    matchNames:
      - consul
  selector:
    matchLabels:
      app: consul
      component: ingress-gateway
  podMetricsEndpoints:
  - port: metrics

To make a ServiceMonitor to work, the service templates need to be adjusted similar. And for the termination gateway a service template needs to be created afaik.

• https://github.com/hashicorp/consul-k8s/blob/e4776cedd2f941e21e7fc65a36a0730ccdcd0966/charts/consul/templates/ingress-gateways-service.yaml#L37
• https://github.com/hashicorp/consul-k8s/blob/e4776cedd2f941e21e7fc65a36a0730ccdcd0966/charts/consul/templates/mesh-gateway-service.yaml#L22

Use Case(s)

Allow to scrape metrics endpoint from gateways using a PodMonitor in RedHat OpenShift using the monitoring stack from OCP.

Contributions

If needed, yes.