helm-charts icon indicating copy to clipboard operation
helm-charts copied to clipboard

Document OIDC settings for popular IDPs

Open nitrocode opened this issue 1 year ago • 3 comments

For instance, okta.

I have this ingress group setup for aws load balancer controller which is handy to allow oidc authentication directly to the atlantis load balancer

One `Ingress` for the webhooks, part of a single `IngressGroup`
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    # only needed if using a group
    alb.ingress.kubernetes.io/group.name: atlantis
    alb.ingress.kubernetes.io/group.order: "1"
    # These are all needed
    alb.ingress.kubernetes.io/actions.github-ipv4-webhooks: '{"forwardConfig":{"targetGroups":[{"serviceName":"atlantis","servicePort":"4141","weight":100}]},"type":"forward"}'
    alb.ingress.kubernetes.io/actions.github-ipv6-webhooks: '{"forwardConfig":{"targetGroups":[{"serviceName":"atlantis","servicePort":"4141","weight":100}]},"type":"forward"}'
    alb.ingress.kubernetes.io/certificate-arn: <snip>
    alb.ingress.kubernetes.io/conditions.github-ipv4-webhooks: '[{"field":"source-ip","sourceIpConfig":{"values":["192.30.252.0/22","185.199.108.0/22","140.82.112.0/20","143.55.64.0/20"]}}]'
    alb.ingress.kubernetes.io/conditions.github-ipv6-webhooks: '[{"field":"source-ip","sourceIpConfig":{"values":["2a0a:a440::/29","2606:50c0::/32"]}}]'
    alb.ingress.kubernetes.io/inbound-cidrs: 0.0.0.0/0
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTP":80},{"HTTPS":443}]'
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/ssl-redirect: "443"
    external-dns.alpha.kubernetes.io/hostname: atlantis.org.com
    kubernetes.io/ingress.class: alb
  creationTimestamp: "2024-01-12T18:06:03Z"
  finalizers:
  - group.ingress.k8s.aws/atlantis
  generation: 1
  name: atlantis-webhooks
  namespace: atlantis
spec:
  rules:
  - http:
      paths:
      - backend:
          service:
            name: github-ipv4-webhooks
            port:
              name: use-annotation
        pathType: ImplementationSpecific
  - http:
      paths:
      - backend:
          service:
            name: github-ipv6-webhooks
            port:
              name: use-annotation
        pathType: ImplementationSpecific
One `Ingress` for oidc, part of a single `IngressGroup`
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    # only needed if using a group
    alb.ingress.kubernetes.io/group.name: atlantis
    alb.ingress.kubernetes.io/group.order: "2"
    # These are all needed
    alb.ingress.kubernetes.io/auth-idp-oidc: '{"authorizationEndpoint":"https://org.okta.com/oauth2/v1/authorize","issuer":"https://org.okta.com","secretName":"atlantis-oidc","tokenEndpoint":"https://org.okta.com/oauth2/v1/token","userInfoEndpoint":"https://org.okta.com/oauth2/v1/userinfo"}'
    alb.ingress.kubernetes.io/auth-on-unauthenticated-request: authenticate
    alb.ingress.kubernetes.io/auth-scope: openid profile
    alb.ingress.kubernetes.io/auth-session-cookie: AWSELBAuthSessionCookie
    alb.ingress.kubernetes.io/auth-session-timeout: "86400"
    alb.ingress.kubernetes.io/auth-type: oidc
    alb.ingress.kubernetes.io/certificate-arn: <snip>
    alb.ingress.kubernetes.io/inbound-cidrs: 0.0.0.0/0
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTP":80},{"HTTPS":443}]'
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/ssl-redirect: "443"
    external-dns.alpha.kubernetes.io/hostname: atlantis.org.com
    kubernetes.io/ingress.class: alb
  creationTimestamp: "2024-01-12T18:11:42Z"
  finalizers:
  - group.ingress.k8s.aws/atlantis
  generation: 2
  name: atlantis-oidc
  namespace: default
spec:
  defaultBackend:
    service:
      name: atlantis
      port:
        number: 4141

Hope that helps someone in the future

nitrocode avatar Jan 12 '24 20:01 nitrocode

Great resource, thank you @nitrocode for putting this together.

One caveat, the service template today doesn't allow for port names and will error on usage of a string (i.e use-annotations). The helm chart needs to be amended to allow this configuration before relevant documentation is added

ri-roee avatar May 01 '24 19:05 ri-roee

Good call @ri-roee. If you have a better working configuration, please post it because I have a feeling I may run into that same issue soon.

nitrocode avatar May 02 '24 16:05 nitrocode

Not a great workaround but I basically took your code and converted it into a kubernetes_ingress_v1 TF object:

resource "kubernetes_ingress_v1" "webhook_ingress" {
  metadata {
    name      = "atlantis-webhook"
    namespace = local.atlantis_ns
    annotations = {
      "alb.ingress.kubernetes.io/actions.github-ipv4-webhooks"    = "{\"forwardConfig\":{\"targetGroups\":[{\"serviceName\":\"atlantis\",\"servicePort\":\"80\",\"weight\":100}]},\"type\":\"forward\"}"
      "alb.ingress.kubernetes.io/actions.github-ipv6-webhooks"    = "{\"forwardConfig\":{\"targetGroups\":[{\"serviceName\":\"atlantis\",\"servicePort\":\"80\",\"weight\":100}]},\"type\":\"forward\"}"
      "alb.ingress.kubernetes.io/certificate-arn"                 = <cert-here>
      "alb.ingress.kubernetes.io/conditions.github-ipv4-webhooks" = "[{\"field\":\"source-ip\",\"sourceIpConfig\":{\"values\":[\"192.30.252.0/22\",\"185.199.108.0/22\",\"140.82.112.0/20\",\"143.55.64.0/20\"]}}]"
      "alb.ingress.kubernetes.io/conditions.github-ipv6-webhooks" = "[{\"field\":\"source-ip\",\"sourceIpConfig\":{\"values\":[\"2a0a:a440::/29\",\"2606:50c0::/32\"]}}]"
      "alb.ingress.kubernetes.io/group.name"                      = "atlantis"
      "alb.ingress.kubernetes.io/group.order"                     = "1"
      "alb.ingress.kubernetes.io/scheme"                          = "internet-facing"
      "alb.ingress.kubernetes.io/ssl-redirect"                    = "443"
      "alb.ingress.kubernetes.io/target-type"                     = "ip"
      "external-dns.alpha.kubernetes.io/hostname"                 = "<domain here>"
      "kubernetes.io/ingress.class"                               = "alb"
    }
  }

  spec {
    rule {
      http {
        path {
          backend {
            service {
              name = "github-ipv4-webhooks"
              port {
                name = "use-annotation"
              }
            }
          }
          path_type = "ImplementationSpecific"
        }

        path {
          backend {
            service {
              name = "github-ipv6-webhooks"
              port {
                name = "use-annotation"
              }
            }
          }
          path_type = "ImplementationSpecific"

        }
      }
    }
  }
}

ri-roee avatar May 02 '24 16:05 ri-roee