dask-gateway icon indicating copy to clipboard operation
dask-gateway copied to clipboard

Image pull secrets not working from a subchart

Open erl987 opened this issue 4 years ago • 1 comments

Environment:

  • dask-gateway: 0.8.0
  • helm: version.BuildInfo{Version:"v3.5.2", GitCommit:"167aac70832d3a384f65f9745335e9fb40169dc2", GitTreeState:"dirty", GoVersion:"go1.15.7"}
  • kubectl: Client Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.2", GitCommit:"faecb196815e248d3ecfb03c680a4507229c2a56", GitTreeState:"clean", BuildDate:"2021-01-13T13:28:09Z", GoVersion:"go1.15.5", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"17+", GitVersion:"v1.17.9-r0-CCE20.8.1.3.B001-17.36.5", GitCommit:"d61c11191a704802f93e5d86877a8fcffe208d58", GitTreeState:"clean", BuildDate:"2020-10-16T09:38:58Z", GoVersion:"go1.13.15", Compiler:"gc", Platform:"linux/amd64"}
  • OS: Debian 10

Bug:

Image pull secrets are ignored if the Helm chart is a subchart of some other chart. This happens at least for the gateway and controller deployments.

It works though if the helm chart is directly installed (with the same values.yaml file configuration) as described here: https://gateway.dask.org/install-kube.html#install-the-helm-chart

Private repositories are relevant today because of the Docker Hub login limits.

Steps to reproduce:

  • A Helm chart demo-cluster with the following configuration: Chart.yaml

    apiVersion: v2
    name: demo-cluster
    description: A cluster
    
    type: application
    version: 0.1.0
    appVersion: 0.1.0
    
    dependencies:
      - name: dask-gateway
        repository: "https://dask.org/dask-gateway-helm-repo/"
        version: 0.8.0
    

    values.yaml

    dask-gateway:
      gateway:
        image:
          name: xxx.xxx.xxx.xxx:xxxxx/daskgateway/dask-gateway-server
          tag: 0.8.0
          pullPolicy: Always
    
        imagePullSecrets:
          - name: default-secret
    
      controller:
        image:
          name: xxx.xxx.xxx.xxx:xxxxx/daskgateway/dask-gateway-server
          tag: 0.8.0
          pullPolicy: Always
    
        imagePullSecrets:
          - name: default-secret
    
  • The dask-gateway-server needs to be uploaded to some private repository.

  • Install the cluster with:

    helm dependency update ./demo-cluster
    helm install acluster ./demo-cluster/
    
  • Now the pods will fail with ImagePullBackOff and the logs reveal that this is because they are missing the image pull secret.

  • This can also be seen by the output of helm install acluster ./demo-cluster/ --dry-run --debug:

    # Source: demo-cluster/charts/dask-gateway/templates/controller/deployment.yaml
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: controller-acluster-dask-gateway
      labels:
        app.kubernetes.io/name: dask-gateway
        helm.sh/chart: dask-gateway-0.8.0
        app.kubernetes.io/instance: acluster
        app.kubernetes.io/version: "0.8.0"
        app.kubernetes.io/managed-by: Helm
        gateway.dask.org/instance: acluster-dask-gateway
    spec:
      replicas: 1
      strategy:
        type: Recreate
      selector:
        matchLabels:
          app.kubernetes.io/name: dask-gateway
          app.kubernetes.io/instance: acluster
          app.kubernetes.io/component: controller
      template:
        metadata:
          labels:
            app.kubernetes.io/name: dask-gateway
            helm.sh/chart: dask-gateway-0.8.0
            app.kubernetes.io/instance: acluster
            app.kubernetes.io/version: "0.8.0"
            app.kubernetes.io/managed-by: Helm
            gateway.dask.org/instance: acluster-dask-gateway
            app.kubernetes.io/component: controller
          annotations:
            checksum/configmap: 6c7fe4e12fc5e497efb932e8611fdb5a6fdb582cb4913c6b903fe1fbdb833196
        spec:
          serviceAccountName: controller-acluster-dask-gateway
          volumes:
            - name: configmap
              configMap:
                name: controller-acluster-dask-gateway
          containers:
            - name: controller
              image: xxx.xxx.xxx.xxx:xxxxx/daskgateway/dask-gateway-server:0.8.0
              imagePullPolicy: Always
              command:
                - dask-gateway-server
                - kube-controller
                - --config
                - /etc/dask-gateway/dask_gateway_config.py
              resources:
                {}
              volumeMounts:
                - mountPath: /etc/dask-gateway/
                  name: configmap
              ports:
                - containerPort: 8000
                  name: api
    

    There is no imagePullSecrets entry which is present in the same deployment if installing the Helm chart directly.

erl987 avatar Feb 19 '21 11:02 erl987

I can't reproduce using helm v3.5.3 and the exact configuration above. Here is what I did:

helm create demo-cluster
rm demo-cluster/templates/ -rf
vim demo-cluster/Chart.yaml # put here the content of the description
vim demo-cluster/values.yaml # put here the content of the description
helm dependency update ./demo-cluster
helm install acluster ./demo-cluster/ --dry-run --debug 2>/dev/null | grep imagePullSecrets # Returns 3 lines

Id2ndR avatar Apr 20 '21 16:04 Id2ndR