Unable to deploy nginx to consul-k8s when connect inject is enabled.

[GeorgeJose7]

Overview of the Issue

I am try to deploy consul in a digital ocean Kubernetes cluster using helm. Once consul is up and running I am trying to deploy nginx into the cluster. The pods for nginx are not starting up. and the follow information was found in the logs.

Defaulted container "consul-dataplane" out of: consul-dataplane, nginx-deployment, consul-connect-inject-init (init)
Error from server (BadRequest): container "consul-dataplane" in pod "nginx-deployment-68d9dc9859-dgqbj" is waiting to start: PodInitializing

How can I resolve this issue?

Kubernetes information

Provider : Digital ocean version : 1.30.2-do.0

Consul helm Chart

chart information

annotations:
  artifacthub.io/images: |
    - name: consul
      image: hashicorp/consul:1.19.1
    - name: consul-k8s-control-plane
      image: hashicorp/consul-k8s-control-plane:1.5.1
    - name: consul-dataplane
      image: hashicorp/consul-dataplane:1.5.1
    - name: envoy
      image: envoyproxy/envoy:v1.25.11
  artifacthub.io/license: MPL-2.0
  artifacthub.io/links: |
    - name: Documentation
      url: https://www.consul.io/docs/k8s
    - name: hashicorp/consul
      url: https://github.com/hashicorp/consul
    - name: hashicorp/consul-k8s
      url: https://github.com/hashicorp/consul-k8s
  artifacthub.io/prerelease: "false"
  artifacthub.io/signKey: |
    fingerprint: C874011F0AB405110D02105534365D9472D7468F
    url: https://keybase.io/hashicorp/pgp_keys.asc
apiVersion: v2
appVersion: 1.19.1
description: Official HashiCorp Consul Chart
home: https://www.consul.io
icon: https://raw.githubusercontent.com/hashicorp/consul-k8s/main/assets/icon.png
kubeVersion: '>=1.22.0-0'
name: consul
sources:
- https://github.com/hashicorp/consul
- https://github.com/hashicorp/consul-k8s
version: 1.5.1

Helm chart values

global:
  name: consul
  enabled: true
  datacenter: dc1
  tls:
    enabled: true
    enableAutoEncrypt: true
    verify: true
  acls: 
    manageSystemACLs: true
    gossipEncryption:
      secretName: consul-bootstrap-secret
      secretKey: token

# Disable the expose server in production.
server:
  replicas: 1
  bootstrapExpect: 1
  exposeService:
    enabled: false
  #  type: LoadBalancer
  storage: 5Gi
  storageClass: do-block-storage
  resources:
    requests:
      memory: "100Mi"
      cpu: "100m"
    limits:
      memory: "100Mi"
      cpu: "100m"

connectInject:
  enabled: true
  default: true
  k8sAllowNamespaces: ['*']
#  aclInjectToken:
#    secretName: consul-bootstrap-secret
#    secretKey: token
  apiGateway:
    managedGatewayClass:
      serviceType: LoadBalancer
  

meshGateway:
  enabled: false
  replicas: 1

controller:
  enabled: true

ui:
  enabled: true
  service:
    enabled: true
    type: LoadBalancer

terminatingGateways:
  enabled: true

API Gateway

apiVersion: gateway.networking.k8s.io/v1beta1
# The Gateway is the main infrastructure resource that links API gateway components.
kind: Gateway
metadata:
 name: api-gateway
 namespace: consul
spec:
 gatewayClassName: consul
 # Configures the listener that is bound to the gateway's address.
 listeners:
   # Defines the listener protocol (HTTP, HTTPS, or TCP)
 - protocol: HTTPS
   port: 443
   name: https
   allowedRoutes:
     namespaces:
       from: All
   tls:
     # Defines the certificate to use for the HTTPS listener.
     certificateRefs:
       - name: consul-server-cert
         kind: Secret

Nginx manifest

# Deployment
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: nginx-deployment
  name: nginx-deployment
  namespace: default
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx-deployment
  template:
    metadata:
      labels:
        app: nginx-deployment
      annotations:
        'consul.hashicorp.com/connect-inject': 'true'
    spec:
      serviceAccountName: nginx-sa
      containers:
      - image: k8s.gcr.io/ingressconformance/echoserver:v0.0.1
        name: nginx-deployment
        env:
        - name: SERVICE_NAME
          value: nginx-deployment
        - name: POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        ports:
        - containerPort: 80
---
# service
apiVersion: v1
kind: Service
metadata:
  labels:
    app: nginx-service
  name: nginx-service
  namespace: default
spec:
  ports:
  - port: 443
    protocol: TCP
    targetPort: 80
  selector:
    app: nginx-deployment
---
# service intention
apiVersion: consul.hashicorp.com/v1alpha1
kind: ServiceIntentions
metadata:
  name: api-gateway 
spec:
  destination:
    name: nginx-service
  sources:
    - name: api-gateway
      action: allow  
---
# service defaults
apiVersion: consul.hashicorp.com/v1alpha1
kind: ServiceDefaults
metadata:
  name: nginx-service-default
  namespace: default
spec:
  protocol: http
---
# service account
apiVersion: v1
kind: ServiceAccount
metadata:
  name: nginx-sa
  namespace: default
automountServiceAccountToken: true
---
# http route
apiVersion: gateway.networking.k8s.io/v1beta1
kind: HTTPRoute
metadata:
  name: route-echo
  namespace: default
spec:
  parentRefs:
  - name: api-gateway
    namespace: consul
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /echo
    backendRefs:
    - kind: Service
      name: nginx-service