litmus icon indicating copy to clipboard operation
litmus copied to clipboard

Support for containerd container runtime for DNS experiments(pod dns spoof and pod dns error)

Open pawanphalak opened this issue 3 years ago • 1 comments
trafficstars

Hi team,

We tried executing the DNS experiments on a GKE cluster with containerd container runtime.But the experiments helper pods are failing with the below error on helper pods: level=error msg="[cri]: Failed to run crictl: time=\"2022-09-15T05:59:22Z\" level=fatal msg=\"failed to connect: failed to connect, make sure you are running as root and the runtime has been started: context deadline exceeded\"\n"

The dns experiment documentation(https://litmuschaos.github.io/litmus/experiments/categories/pods/pod-dns-error/#container-runtime-socket-path) mentions that container runtime for dns experiments is only supported for docker. Can you please share when will the dns experiments suport containerd runtime or if there is any existing workaround to run the DNS chaos experiments in GKE cluster with containerd?

pawanphalak avatar Sep 15 '22 06:09 pawanphalak

UPDATE:

I have added the one more parameter for chaosengine SOCKET_PATH and now helper pods are returning the below error: level=fatal msg="Failed to create Interceptor" error="failed to inject interceptor in resolv.conf : open /etc/resolv.conf: read-only file system" level=fatal msg="Failed to run command" cmd=/usr/local/bin/dns_interceptor error="exit status 1"

Following is the cronschedule I used to run the pod-dns-error chaos:

apiVersion: litmuschaos.io/v1alpha1
kind: ChaosSchedule
metadata:
  name: schedule-pod-dns-error
  namespace: family-an-1s1-perf
spec:
  schedule:
    repeat:
      properties:
        minChaosInterval:
          # schedule the chaos at every 1 minutes
          minute:
            everyNthMinute: 1  
  engineTemplateSpec:
    engineState: 'active'
    annotationCheck: 'false'
    components:
      runner:
        runnerAnnotations:
          sidecar.istio.io/inject: "false"
        # resource requirements for the runner pod
        resources:
          requests:
            cpu: "50m"
            memory: "64Mi"
          limits:
            cpu: "100m"
            memory: "128Mi"
    appinfo:
      appns: "family-an-1s1-perf"
      applabel: "app=productpage"
      appkind: "deployment"
    chaosServiceAccount: pod-dns-error-sa
    jobCleanUpPolicy: 'delete'
    experiments:
      - name: pod-dns-error
        spec:
          components:
            experimentAnnotations:
              sidecar.istio.io/inject: "false"
            env:
              - name: TARGET_HOSTNAMES
                value: '["details.family-an-1s1-perf.svc.cluster.local"]'
              - name: TOTAL_CHAOS_DURATION
                value: '120'
              - name: 'PODS_AFFECTED_PERC'
                value: '100'
              - name: TARGET_CONTAINER
                value: 'productpage'
              - name: CONTAINER_RUNTIME
                value: 'containerd'
              - name: SOCKET_PATH
                value: '/run/containerd/containerd.sock'

Any suggestions on how this can be resolved or how can we configure dns chaos for containerd runtime?

pawanphalak avatar Sep 15 '22 09:09 pawanphalak

I wouldn't classify it as a bug as the documentation clearly shows that only docker runtime is supported for now.

bbarin avatar Oct 04 '22 15:10 bbarin

I've found this PR adding support for containerd for this experiments: https://github.com/litmuschaos/litmus-go/pull/577

dmcardoso avatar Oct 05 '22 17:10 dmcardoso

Closing this issue, as it has been resolved with litmuschaos/litmus-go#577. Thanks everyone for participating!

neelanjan00 avatar Oct 07 '22 05:10 neelanjan00