Ability to customize the dind image in gha-runner-scale-set

[victor-cetinic]

What would you like added?

I have been trying to customise the gha-runner-scale-set so that i can set the docker registry mirror globally for all the runners using the Docker DIND approach.

Most instructions point to modifying the registry settings of the docker service by adding a command line like : --registry-mirror=https://mirror.gcr.io (further options - https://cloud.google.com/artifact-registry/docs/pull-cached-dockerhub-images )

The issue im facing is that these need to be configured on the dind container, and not the runner container., and when i created a template like this

template:
  spec: 
    nodeSelector:
      cloud.google.com/gke-preemptible: "true"
    initContainers:
      - name: dockerconfigwriter
        image: alpine
        command:
          - sh
          - -c
          - |
            echo "{\"registry-mirrors\": [ \"https://mirror.gcr.io\"] }" > /home/runner/.docker/daemon.json
        volumeMounts:
          - name: docker-config-volume
            mountPath: /home/runner/.docker
    containers:
      - name: runner
        command:
          - /home/runner/run.sh
        image:
          ghcr.io/actions/actions-runner:latest
        env:
          - name: DOCKER_HOST
            value: unix:///var/run/docker.sock
          - name: RUNNER_WAIT_FOR_DOCKER_IN_SECONDS
            value: \"120\"
        resources:
          limits:
            cpu: 6
            memory: 5Gi
        volumeMounts:
          - name: work
            mountPath: /home/runner/_work
          - name: dind-sock
            mountPath: /var/run
          - name: docker-config-volume
            mountPath: /etc/docker
      - name: dind
        image: docker:dind
        args:
          - dockerd
          - --host=unix:///var/run/docker.sock
          - --group=$(DOCKER_GROUP_GID)
          - --registry-mirror=https://mirror.gcr.io
        env:
          - name: DOCKER_GROUP_GID
            value: \"123\"
        securityContext:
          privileged: true
        volumeMounts:
          - name: work
            mountPath: /home/runner/_work
          - name: dind-sock
            mountPath: /var/run
          - name: dind-externals
            mountPath: /home/runner/externals
          - name: docker-config-volume
            mountPath: /etc/docker
    volumes:
      - name: docker-config-volume
        emptyDir: { }

I expected to be able to customise the dind container, however, it does not get included in the template, it simply uses the default pre-defined in the template, so none of these customisations are considered in the final template, consequently , the settings are not applied.

Output from dry run

Below you can see the final template, that contains the customisations i made to the runner, however, the dind container is ignored so my mirror registry settings are not applied.

  containers:
      - name: runner
        
        command: 
          - /home/runner/run.sh
        image: 
          ghcr.io/actions/actions-runner:latest
        resources: 
          limits:
            cpu: 6
            memory: 5Gi
        env:
          - 
            name: DOCKER_HOST
            value: unix:///var/run/docker.sock
          - 
            name: RUNNER_WAIT_FOR_DOCKER_IN_SECONDS
            value: \"120f\"
        volumeMounts:
          - 
            mountPath: /home/runner/_work
            name: work
          - 
            mountPath: /var/run
            name: dind-sock
          - 
            mountPath: /etc/docker
            name: docker-config-volume
      - name: dind
        image: docker:dind
        args:
          - dockerd
          - --host=unix:///var/run/docker.sock
          - --group=$(DOCKER_GROUP_GID)
        env:
          - name: DOCKER_GROUP_GID
            value: "123"
        securityContext:
          privileged: true
        volumeMounts:
          - name: work
            mountPath: /home/runner/_work
          - name: dind-sock
            mountPath: /var/run
          - name: dind-externals
            mountPath: /home/runner/externals

Why is this needed?

The issue for me is I hit limits going to docker registry, and having a single way to do this for all runners, makes more sense and would apply for all rather than each workflow having to do something

Additional context

Add any other context or screenshots about the feature request here.

[github-actions[bot]]

Hello! Thank you for filing an issue.

The maintainers will triage your issue shortly.

In the meantime, please take a look at the troubleshooting guide for bug reports.

If this is a feature request, please review our contribution guidelines.

[Alpenovsky]

I had the same problem. I use FluxCD to create Helm release and I used the postRenderers functionality to overwrite (as a workaround) the dind image or dind args:

apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
  name: dev-runners
spec:
  releaseName: dev-runners
  chartRef:
    kind: OCIRepository
    name: dev-runners
  postRenderers:
    - kustomize:
        patches:
          - target:
              kind: AutoscalingRunnerSet
              name: dev-runners
            patch: |
              - op: replace
                path: /spec/template/spec/containers/1/image
                value: 24.0.9-dind
[...]

[orbi-ci-bot]

+1 We are using RunnerDeployments. Can we get this option on there as well. We have reached our pull limit with Docker hub with no way to update the image.

[thomaschaplin]

Would love to see this become a feature. Any update on this?

[norman-zon]

would be possible with https://github.com/actions/actions-runner-controller/pull/3645

[SharifiFaranak]

We would like this feature as well. We would like to set resource requests under the dind container since that is the container that runs the workflows.

We tried using Kustomize patch with ArgoCD to achieve this, but running into another error when trying that way - helm pull --untar --untardir .manifests/runner-scale-sets/dind-large/charts --repo oci://ghcr.io/actions/actions-runner-controller-charts gha-runner-scale-set --version 0.9.0 (helm pull does not support oci with the --repo flag).

Instead of trying to make that work, would love to be able to just use the helm chart here to configure the dind container.

Thanks!

[evilhamsterman]

@nikola-jokic @mhuijgen you two seem to be the maintainers now. This is a very requested feature there are at least 5 PRs just requesting the ability to easily change the DinD image https://github.com/actions/actions-runner-controller/pull/3645 https://github.com/actions/actions-runner-controller/pull/3851 https://github.com/actions/actions-runner-controller/pull/3540 https://github.com/actions/actions-runner-controller/pull/3394 https://github.com/actions/actions-runner-controller/pull/3216

And several issues #3997 #3709 #3513

The logic in a previous discussion https://github.com/actions/actions-runner-controller/pull/3216#issuecomment-2063716555 is kind of sound but fails when compared to other Helm Charts. It also fails because you still can't completely control everything a lot gets overridden anyway #3709

Typically most charts will use a default template and have options for easy overrides of things like image and resources, and options to allow for extras to common features like volumes, containers, labels, annotations. Then if you really need it they may also have an option to completely override the template with the knowledge that you have to watch for changes on update that normally people don't.

some pseudo code like

template:
  spec:
    {{ if .values.template.override }}
      {{ .values.template.override}}
    {{ else }}
      {{ .defaultTemplate }}