Proxy protocol header includes wrong destination port

[JuniorJPDJ]

Description:

• listener set to listen on port 443
• proxy protocol enabled by backendtrafficpolicy
• proxy protocol header to the backend contains 10443 port

This is probably due to envoy actually listening on 10443 port and service rewriting it to 443

Repro steps:

apiVersion: gateway.envoyproxy.io/v1alpha1
kind: BackendTrafficPolicy
metadata:
  name: proxy-protocol-route-knyfyrtel-ingress-tls
  namespace: knyfyrtel
spec:
  proxyProtocol:
    version: V2
  targetRef:
    group: gateway.networking.k8s.io
    kind: TLSRoute
    name: knyfyrtel-ingress-tls
---
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: default
  namespace: envoy-gateway-system
spec:
  gatewayClassName: envoy-gateway
  listeners:
    - allowedRoutes:
        namespaces:
          from: All
      name: http-0
      port: 80
      protocol: HTTP
    - allowedRoutes:
        namespaces:
          from: All
      name: tls-pass-0
      port: 443
      protocol: TLS
      tls:
        mode: Passthrough
---
apiVersion: gateway.networking.k8s.io/v1alpha2
kind: TLSRoute
metadata:
  name: knyfyrtel-ingress-tls
  namespace: knyfyrtel
spec:
  hostnames:
    - <REDACTED>
  parentRefs:
    - group: gateway.networking.k8s.io
      kind: Gateway
      name: default
      namespace: envoy-gateway-system
      sectionName: tls-pass-0
  rules:
    - backendRefs:
        - group: ''
          kind: Service
          name: ingress-nginx-controller-x-ingress-nginx-x-knyfyrtel-vcluster
          port: 443
          weight: 1

tshark -i - --disable-protocol tls -V -Y "proxy.src.ipv4"

Environment: Envoy Gateway Helm Chart deployment 1.4.1

[shiavm006]

I've worked on resolving this issue by preventing port shifting for TLS passthrough on port 443. This should ensure PROXY protocol headers show the correct destination port (443) instead of the shifted port (10443).

Would you mind reviewing the solution in PR #6427 and sharing your thoughts on whether this addresses your use case?

[JuniorJPDJ]

I'm personally using Proxy protocol also on other types of listeners (eg. plain HTTP). What about just a config value to disable shifting?

[shiavm006]

I'm personally using Proxy protocol also on other types of listeners (eg. plain HTTP). What about just a config value to disable shifting?

i have taken ur point in mind and ya ur point is absoultuly great i have made and implemeted chnages with respect to it now u can review it and let me know

[arkodg]

@JuniorJPDJ the Envoy Proxy pods runs with minimum privileges w/o CAP_NET_BIND_SERVICE, so there's a default 1024 shift for privileged ports. This can be undone by setting useListenerPortAsContainerPort in the EnvoyProxy resource https://gateway.envoyproxy.io/docs/api/extension_types/#envoyproxykubernetesprovider. You may also need to add the capability in the securityContext

[arkodg]

for this specific case, the internal detail is exposed to the backend, which is not great, above workaround should resolve this, lets keep this open to find a better ootb solution

[shiavm006]

You're spot on about the root cause - the 1024 port shift is exactly what's breaking PROXY protocol headers. While the useListenerPortAsContainerPort workaround would work, I've implemented a more user-friendly solution that automatically handles this when PROXY protocol is enabled via ClientTrafficPolicy. The fix detects PROXY protocol and skips port shifting for those cases, so backends get the correct port numbers (443 instead of 10443) without users needing to configure anything extra or add security capabilities. what r ur views on it ?

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had activity in the last 30 days.

[JuniorJPDJ]

not stale

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had activity in the last 30 days.

[guoard]

@JuniorJPDJ the Envoy Proxy pods runs with minimum privileges w/o CAP_NET_BIND_SERVICE, so there's a default 1024 shift for privileged ports. This can be undone by setting useListenerPortAsContainerPort in the EnvoyProxy resource https://gateway.envoyproxy.io/docs/api/extension_types/#envoyproxykubernetesprovider. You may also need to add the capability in the securityContext

Adding the NET_BIND_SERVICE capability didn’t resolve the issue (even granting all capabilities didn’t help). I was able to fix it by setting runAsUser: 0 in the securityContext.

If this is the expected behavior, could you please document it under the Customizing EnvoyProxy section?

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had activity in the last 30 days.

[JuniorJPDJ]

not stale

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had activity in the last 30 days.

[JuniorJPDJ]

I didn't need to set any additional caps or change running user on my setup - I just set useListenerPortAsContainerPort and it worked. Don't know why.