helm-charts icon indicating copy to clipboard operation
helm-charts copied to clipboard

Kestra standalone startup fails due to issue with DIND on Windows

Open darkedges opened this issue 10 months ago • 1 comments

Expected Behavior

Pod should start up with no issues

Actual Behaviour

Pod fails to start with

Device "ip_tables" does not exist.
modprobe: can't change directory to '/lib/modules': No such file or directory
iptables v1.8.10 (nf_tables)
[WARN  tini (99)] Tini is not running as PID 1 and isn't registered as a child subreaper.
Zombie processes will not be re-parented to Tini, so zombie reaping won't work.
To fix the problem, use the -s option or set the environment variable TINI_SUBREAPER to register Tini as a child subreaper, or run Tini as PID 1.
failed to start daemon: Error initializing network controller: error obtaining controller instance: failed to register "bridge" driver: unable to add return rule in DOCKER-ISOLATION-STAGE-1 chain:  (iptables failed: iptables --wait -A DOCKER-ISOLATION-STAGE-1 -j RETURN: iptables v1.8.10 (nf_tables):  RULE_APPEND failed (No such file or directory): rule in chain DOCKER-ISOLATION-STAGE-1
 (exit status 4))
[rootlesskit:child ] error: command [docker-init -- dockerd --host=unix:///dind//docker.sock --host=tcp://0.0.0.0:2376 --tlsverify --tlscacert /certs/server/ca.pem --tlscert /certs/server/cert.pem --tlskey /certs/server/key.pem --log-level=fatal --group=1000] exited: exit status 1
[rootlesskit:parent] error: child exited: exit status 1

Steps To Reproduce

  1. create values.yaml containing

    deployments:
      standalone:
        enabled: true
    
  2. execute using helm install kestra kestra/kestra -n kestra --create-namespace -f .\values.yaml

  3. See

    NAME                                 READY   STATUS    RESTARTS      AGE
    kestra-minio-7fdfd75b8c-27f2f        1/1     Running   0             62m
    kestra-postgresql-0                  1/1     Running   0             62m
    kestra-standalone-59b5f7bbb8-25bnc   1/2     CrashLoopBackOff   4 (19s ago)   2m15s
    

    See pod has failed to start with a CrashBackoff. Log contains

    Device "ip_tables" does not exist.
    modprobe: can't change directory to '/lib/modules': No such file or directory
    iptables v1.8.10 (nf_tables)
    [WARN  tini (99)] Tini is not running as PID 1 and isn't registered as a child subreaper.
    Zombie processes will not be re-parented to Tini, so zombie reaping won't work.
    To fix the problem, use the -s option or set the environment variable TINI_SUBREAPER to register Tini as a child subreaper, or run Tini as PID 1.
    failed to start daemon: Error initializing network controller: error obtaining controller instance: failed to register "bridge" driver: unable to add return rule in DOCKER-ISOLATION-STAGE-1 chain:  (iptables failed: iptables --wait -A DOCKER-ISOLATION-STAGE-1 -j RETURN: iptables v1.8.10 (nf_tables):  RULE_APPEND failed (No such file or directory): rule in chain DOCKER-ISOLATION-STAGE-1
    (exit status 4))
    [rootlesskit:child ] error: command [docker-init -- dockerd --host=unix:///dind//docker.sock --host=tcp://0.0.0.0:2376 --tlsverify --tlscacert /certs/server/ca.pem --tlscert /certs/server/cert.pem --tlskey /certs/server/key.pem --log-level=fatal --group=1000] exited: exit status 1
    [rootlesskit:parent] error: child exited: exit status 1
    
  4. update 1values.yaml` to include

    dind:
     image:
       tag: dind
     args:
       - --log-level=fatal
     securityContext:
       runAsUser: 0
       runAsGroup: 0
    
    securityContext:
     runAsUser: 0
     runAsGroup: 0
    
  5. upgrade using helm upgrade kestra kestra/kestra -n kestra --create-namespace -f .\values.yaml

  6. Check logs.

    Certificate request self-signature ok
    subject=CN = docker:dind server
    /certs/server/cert.pem: OK
    Certificate request self-signature ok
    subject=CN = docker:dind client
    /certs/client/cert.pem: OK
    iptables v1.8.10 (nf_tables)
    failed to start daemon: Error initializing network controller: error obtaining controller instance: failed to register "bridge" driver: 
    unable to add return rule in DOCKER-ISOLATION-STAGE-1 chain:  (iptables failed: iptables --wait -A DOCKER-ISOLATION-STAGE-1 -j RETURN: iptables v1.8.10 (nf_tables):  RULE_APPEND failed (No such file or directory): rule in chain DOCKER-ISOLATION-STAGE-1
     (exit status 4))
    
  7. update values.yaml to include

    dind:
     image:
       tag: stable-dind
     args:
       - --log-level=fatal
     securityContext:
       runAsUser: 0
       runAsGroup: 0
    
    securityContext:
     runAsUser: 0
     runAsGroup: 0
    
  8. pod starts without any issues

    NAME                                 READY   STATUS    RESTARTS   AGE
    kestra-minio-7fdfd75b8c-27f2f        1/1     Running   0          59m
    kestra-postgresql-0                  1/1     Running   0          59m
    kestra-standalone-7b798467d7-m44g2   2/2     Running   0          45s
    

Environment Information

  • Kestra Version: latest
  • Helm Charts version: latest
  • Docker Image version: latest

Window 11 - Docker in Desktop

darkedges avatar Apr 01 '24 21:04 darkedges

Hi ! Tried to reproduce your issue and couldn't

I have tried on Windows 11 with a fresh install of kube cli/helm/minikube (through chocolatey), copy/paste your commands and its run fine Do you still face the issue?

Skraye avatar Jul 31 '24 07:07 Skraye