Session Affinity

[maxpain]

trafficstars

Do Argo Tunnels supports Session Affinity (sticky sessions) in cluster scope? For example, Nginx Ingress Controller allows specifying several annotations for it:

nginx.ingress.kubernetes.io/session-cookie-name: "__lbroute"
nginx.ingress.kubernetes.io/session-cookie-hash: "sha1"

It's important to us, due we support old web-browsers without WebSockets, using XHR Long Polling as a fallback.

[mattalberts]

@Maxpain177 the tunnel controller doesn't expose a way to configure sticky sessions, though you are able to configure session affinity as part of the load-balancer

• LoadBalancer Setup
• LoadBalancer Session Affinity Setup

[maxpain]

@mattalberts, Yes, I think about "Session affinity" feature as part of the CloudFlare Load Balancer. But it associates all user's requests with a single origin (i.e., Kubernetes Node in my case). I need to associate every request not only with single origin (Node) but with single Endpoint/Pod.

[mattalberts]

@Maxpain177 (sorry for the delay was digging around/investigating/fixing issues).

Correct, the current version of the ingress controller maps a "tunnel" to a service (specifically namespace.name:port). I that model, you're left with the load-balancing options defined by a service (or rather the the configured proxy mode, likely ipvs or iptables).

• https://kubernetes.io/docs/concepts/services-networking/

Have you tried setting sessionAffinity to ClientIP on your service? Of course, that is still only clientIp based affinity ... not cookie (which is actually what the cloudflare load balancers provide). I think the solution to this will be to map a "tunnel" per endpoint, allowing the load-balancer to route by session cookie.

[maxpain]

@mattalberts, Hi. ClientIP sessionAffinity does not evenly load balances when multiple clients seat behind NAT (with same IPs). It will be cool to use Endpoints API (like ingress-nginx) instead of Service and make load balancing with Cookie session affinity.

[mattalberts]

@Maxpain177 correct. As i try to draft up changes for 0.6.x I can start looking at that. An easy way around the restrictions (and maybe a better model than a centralized controller) would be to run cloudflared as a side-car container (part of your replicaset spec template used to spin up pods). That would give you a cloudflared instance per-pod. The solutions does come with some downsides that the controller end-arounds, but.

[mattalberts]

@Maxpain177

To ensure you don't get blocked or sad, I thought I'd update the issue. I don't have plans to correct the ingress-controller in the short-term (or even mid-term). Given your use-case, I believe you should take a different route, and deploy cloudflared as a side-car container (e.g. do not use the ingress-controller at all).

Deploying cloudflared as a side-car will allow you to directly leverage sticky routing at the cloudflare load-balancer.

apiVersion: v1
kind: Service
metadata:
  labels:
    app: echo
  name: echo
spec:
  ports:
  - name: http
    port: 80
    protocol: TCP
    targetPort: 8080
  selector:
    app: echo
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  labels:
    app: echo
  name: echo
spec:
  replicas: 2
  selector:
    matchLabels:
      app: echo
  template:
    metadata:
      labels:
        app: echo
    spec:
      containers:
      - name: echo
        image: k8s.gcr.io/echoserver:1.10
        imagePullPolicy: Always
        env:
        - name: NODE_NAME
          valueFrom:
            fieldRef:
              fieldPath: spec.nodeName
        - name: POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        - name: POD_IP
          valueFrom:
            fieldRef:
              fieldPath: status.podIP
        ports:
        - containerPort: 8080
        resources:
          limits:
            cpu: 20m
            memory: 40Mi
          requests:
            cpu: 20m
            memory: 40Mi
      - name: tunnel
        image: docker.io/mattalberts/cloudflared-linux-amd64:2018.11.0
        imagePullPolicy: Always
        command: ["cloudflared", "tunnel"]
        args:
        - --url=http://127.0.0.1:8080
        - --hostname=echo.mydomain.com
        - --lb-pool=echo
        - --loglevel=debug
        - --origincert=/etc/cloudflared/cert.pem
        - --proto-loglevel=warn
        - --no-autoupdate
        - --retries=8
        - --tag=app=echo,src=k8s,name=$(POD_NAME),namespace=$(POD_NAMESPACE),host=echo.mattalberts.pink
        env:
        - name: POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        resources:
          limits:
            cpu: 10m
            memory: 20Mi
          requests:
            cpu: 100m
            memory: 64Mi
        volumeMounts:
        - mountPath: /etc/cloudflared
          name: tunnel-secret
          readOnly: true
      terminationGracePeriodSeconds: 60
      volumes:
      - name: tunnel-secret
        secret:
          secretName: mydomain.com
---

notes:

• docker.io/mattalberts/cloudflared-linux-amd64:2018.11.0 was only built for evaluation, please do not use it or rely on its existence (I will be deleting the image)
• you may need to tune the requested resources.