terraform-provider-kubernetes icon indicating copy to clipboard operation
terraform-provider-kubernetes copied to clipboard

Ingress failing after upgrade from kubernetes_ingress to kubernetes_ingress_v1

Open swhiteCQC opened this issue 2 years ago • 1 comments

Terraform version, Kubernetes provider version and Kubernetes version

Terraform version: 0.15.5
Kubernetes Provider version: 2.7.1
Kubernetes version: 1.22.6

Terraform configuration

resource "kubernetes_ingress_v1" "some_name" {
  metadata {
    name      = "some_name"
    namespace = var.namespace
    annotations = {
      "cert-manager.io/issuer"                         = local.cert_manager_issuer
      "nginx.ingress.kubernetes.io/use-regex"          = "true"
      "nginx.ingress.kubernetes.io/rewrite-target"     = "/$1"
      "nginx.ingress.kubernetes.io/proxy-body-size"    = "0"
      "nginx.ingress.kubernetes.io/proxy-read-timeout" = "600"
      "nginx.ingress.kubernetes.io/proxy-send-timeout" = "600"
      "nginx.ingress.kubernetes.io/server-snippet"     = <<-EOT
        location ~* "^/wp-login" {
            deny all;
            return 404;
          }
      EOT
    }
  }

  spec {
    ingress_class_name = "nginx-http${local.nginx_ingress_class_suffix}"
    default_backend {
      service {
        name = module.services.service_name
        port {
          number = 80
        }
      }
    }
    rule {
      host = module.ip.a_fqdn
      http {
        path {
          backend {
            service {
              name = module.services.service_name
              port {
                number = 80
              }
            }
          }
          path = "/(api\\/v5.*)"
        }
        path {
          backend {
            service {
              name = module.services.service_name
              port {
                number = 80
              }
            }
          }
          path = "/(docs.*)"
        }
        path {
          backend {
            service {
              name = module.services.service_name
              port {
                number = 80
              }
            }
          }
          path = "/(openapi\\.json)"
        }
        path {
          backend {
            service {
              name = module.services.service_name
              port {
                number = 80
              }
            }
          }
          path = "/(.*)"
          path_type = "Prefix"
        }
      }
    }

    tls {
      hosts = [
        module.ip.a_fqdn,
      ]
      secret_name = "http-tls-secret"
    }
  }

  depends_on = [module.nginx]
}


resource "kubernetes_ingress_v1" "some_name_b" {
  metadata {
    name      = "some_name_b"
    namespace = var.namespace
    annotations = {
      "cert-manager.io/issuer"                       = local.cert_manager_issuer
      "nginx.ingress.kubernetes.io/backend-protocol" = "GRPC"
    }
  }

  spec {
    ingress_class_name = "nginx-grpc${local.nginx_ingress_class_suffix}"
    rule {
      host = module.ip.b_dn
      http {

        path {
          backend {
            service {
              name = module.services.b_service_name
              port {
                number = 8080
              }
            }
          }
          path = "/"
        }
      }
    }

    tls {
      hosts = [
        module.ip.b_dn,
      ]
      secret_name = "grpc-tls-secret"
    }
  }

  depends_on = [module.nginx]
}

Question

Recently we upgrade our Kubernetes cluster to 1.22.6, which meant that we had to upgrade a number of our resources. One of the areas we upgraded was moving our `kubernetes_ingress` resources to `kubernetes_ingress_v1`.
Applying this upgrade to our existing environments saw no impact on behaviour, but when we try and create a new environment from scratch the ingress is not working as expected.

Each environment should have 2 ingresses, and when we try to build them we see only one of the ingresses is associated with an IP address, and the ingress that does have an IP address doesn't resolve to any backend.
If one of the ingresses is removed the deployed ingress will be associated with an IP address but wont resolve to any backend if the IP address is followed.

There are no unusual events shown by either ingress, the logs from the controllers seem normal and the configuration produced by Terraform appears to be totally normal when inspected. Does anyone have an idea of where to start investigating this issue? I've exhausted the areas that seem like obvious candidates to me.

swhiteCQC avatar May 05 '22 09:05 swhiteCQC

Hi!

are the two ingress objects created with the correct attribute values when you apply your configuration from scratch?

alexsomesan avatar Jul 20 '22 11:07 alexsomesan

Hi, any news on this one? I have the same issue.

eshton avatar Dec 09 '22 15:12 eshton

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

github-actions[bot] avatar Jan 12 '23 02:01 github-actions[bot]