Apllying http-visibility breaks OAuth requests

[pandarun]

trafficstars

Hello, after applying http-visibility

apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
  name: http-visibility
spec:
  endpointSelector:
    matchLabels: {}
  ingress:
    - fromEntities:
        - all
      toPorts:
        - ports:
            - port: "80"
              protocol: TCP
          rules:
            http:
              - {}
    - fromEntities:
        - all

as described in docs makes my services return 401 responses after authorization.

How can I check envoy doesn't trim Authorization headers or Cookies.

NOTE: If I remove the policy everything works as expected.

Any help is appreciated.

[glibsm]

Is it possible your service works on more than just port 80? For example, perhaps it automatically redirects to 443 or something? It's hard to say without seeing more details of the servicec.

Using hubble observe -o json would show you the headers.

You could also print it on the server side as you process the request.

[pandarun]

Is it possible your service works on more than just port 80?

You were right, one of my HTTP services was listenening on non 80 port, but it wasn't the root cause.

The root cause as I found after adding debug prints was that envoy added Content-Length: 0 header for some reason:

{
    "time": "2021-07-18T21:34:31.742188400Z",
    "verdict": "FORWARDED",
    "IP": {
        "source": "10.0.5.42",
        "destination": "10.0.3.171",
        "ipVersion": "IPv4"
    },
    "l4": {
        "TCP": {
            "source_port": 45952,
            "destination_port": 80
        }
    },
    "source": {
        "identity": 6729,
        "namespace": "default",
        "labels": [
            "k8s:app.kubernetes.io/instance=<redacted>-release",
            "k8s:app.kubernetes.io/name=<redacted>",
            "k8s:io.cilium.k8s.namespace.labels.kubernetes.io/metadata.name=default",
            "k8s:io.cilium.k8s.policy.cluster=default",
            "k8s:io.cilium.k8s.policy.serviceaccount=<redacted>-release",
            "k8s:io.kubernetes.pod.namespace=default"
        ],
        "pod_name": "<redacted>-release-7699c7b884-b5r76"
    },
    "destination": {
        "ID": 1248,
        "identity": 21845,
        "namespace": "default",
        "labels": [
            "k8s:app.kubernetes.io/instance=<redacted>-release",
            "k8s:app.kubernetes.io/name=<redacted>",
            "k8s:io.cilium.k8s.namespace.labels.kubernetes.io/metadata.name=default",
            "k8s:io.cilium.k8s.policy.cluster=default",
            "k8s:io.cilium.k8s.policy.serviceaccount=<redacted>-release",
            "k8s:io.kubernetes.pod.namespace=default"
        ],
        "pod_name": "<redacted>-release-7f95449c69-qp2c4"
    },
    "Type": "L7",
    "node_name": "k3s-worker-5",
    "l7": {
        "type": "REQUEST",
        "http": {
            "method": "POST",
            "url": "http://<redacted>-release.default.svc.cluster.local/",
            "protocol": "HTTP/1.1",
            "headers": [
                {
                    "key": "Accept",
                    "value": "text/html, application/xhtml+xml, application/xml; q=0.9, image/webp, */*; q=0.8"
                },
                {
                    "key": "Accept-Encoding",
                    "value": "gzip"
                },
                {
                    "key": "Accept-Language",
                    "value": "en-US, en; q=0.5"
                },            
                **{
                    "key": "Content-Length",
                    "value": "0"
                },**
                {
                    "key": "Content-Type",
                    "value": "application/json"
                },
                {
                    "key": "Cookie",
                    "value": "<redacted>"
                },
                ...
            ]
        }
    },
    "event_type": {
        "type": 129
    },
    "traffic_direction": "INGRESS",
    "is_reply": false,
    "Summary": "HTTP/1.1 POST http://<redacted>-release.default.svc.cluster.local/"
}