Attaching Volume to Kaniko When Building Docker Images using Kaniko In Kubeflow Pipeline

[II-VSB-II]

Hi @Ark-kun , First of all thanks for the sample Notebooks.

I am trying to build Docker Image using Kaniko in my Kubeflow Pipeline as my environment doesn't have Internet access also I need to push Docker Image to private repository so that I can use the image in my next pipeline step.

kfp Version: 0.1.32 Component.yaml

name: Build Image Using Kaniko
description: Stage is for building Docker Image Using Kanio.
implementation:
  container:
    image: gcr.io/kaniko-project/executor:debug
    command: ["busybox/sh", "-c"]
    args: [
      "/kaniko/executor",
      --dockerfile=/data/Dockerfile,
      --context=dir://data,
      --destination=privaterepo/kanikotf:2.2.1,
      --cache=true,
      --skip-tls-verify
    ]
    volumeMounts:
    - name: docker-config 
      mountPath: /kaniko/.docker
    - name: dockerfile-storage 
      mountPath: /data
  restartPolicy: Never
  volumes:
    - name: docker-config 
      secret:
        secretName: docker-config
        items:
          - key: .dockerconfigjson
            path: config.json
    - name: dockerfile-storage
      persistentVolumeClaim: 
        claimName: notebook-volume

While loading Yaml: build_image_op = kfp.components.load_component_from_text(kaniko_yaml)

Error:

TypeError: Error: ContainerImplementation.from_dict(struct=OrderedDict([('container', OrderedDict([('image', 'gcr.io/kaniko-project/executor:debug'), ('command', ['busybox/sh', '-c']), ('args', ['/kaniko/executor', '--dockerfile=/data/Dockerfile', '--context=dir://data', '--destination=privaterepo/kanikotf:2.2.1', '--cache=true', '--skip-tls-verify']), ('volumeMounts', [OrderedDict([('name', 'docker-config'), ('mountPath', '/kaniko/.docker')]), OrderedDict([('name', 'dockerfile-storage'), ('mountPath', '/data')])])])), ('restartPolicy', 'Never'), ('volumes', [OrderedDict([('name', 'docker-config'), ('secret', OrderedDict([('secretName', 'docker-config'), ('items', [OrderedDict([('key', '.dockerconfigjson'), ('path', 'config.json')])])]))]), OrderedDict([('name', 'dockerfile-storage'), ('persistentVolumeClaim', OrderedDict([('claimName', 'notebook-volume')]))])])])) failed with exception:
Error: ContainerSpec.from_dict(struct=OrderedDict([('image', 'gcr.io/kaniko-project/executor:debug'), ('command', ['busybox/sh', '-c']), ('args', ['/kaniko/executor', '--dockerfile=/data/Dockerfile', '--context=dir://data', '--destination=privaterepo/kanikotf:2.2.1', '--cache=true', '--skip-tls-verify']), ('volumeMounts', [OrderedDict([('name', 'docker-config'), ('mountPath', '/kaniko/.docker')]), OrderedDict([('name', 'dockerfile-storage'), ('mountPath', '/data')])])])) failed with exception:
__init__() got an unexpected keyword argument 'volumeMounts'
Error: GraphImplementation.from_dict(struct=OrderedDict([('container', OrderedDict([('image', 'gcr.io/kaniko-project/executor:debug'), ('command', ['busybox/sh', '-c']), ('args', ['/kaniko/executor', '--dockerfile=/data/Dockerfile', '--context=dir://data', '--destination=privaterepo/kanikotf:2.2.1', '--cache=true', '--skip-tls-verify']), ('volumeMounts', [OrderedDict([('name', 'docker-config'), ('mountPath', '/kaniko/.docker')]), OrderedDict([('name', 'dockerfile-storage'), ('mountPath', '/data')])])])), ('restartPolicy', 'Never'), ('volumes', [OrderedDict([('name', 'docker-config'), ('secret', OrderedDict([('secretName', 'docker-config'), ('items', [OrderedDict([('key', '.dockerconfigjson'), ('path', 'config.json')])])]))]), OrderedDict([('name', 'dockerfile-storage'), ('persistentVolumeClaim', OrderedDict([('claimName', 'notebook-volume')]))])])])) failed with exception:
__init__() got an unexpected keyword argument 'container'
Error: Structure "OrderedDict([('container', OrderedDict([('image', 'gcr.io/kaniko-project/executor:debug'), ('command', ['busybox/sh', '-c']), ('args', ['/kaniko/executor', '--dockerfile=/data/Dockerfile', '--context=dir://data', '--destination=privaterepo/kanikotf:2.2.1', '--cache=true', '--skip-tls-verify']), ('volumeMounts', [OrderedDict([('name', 'docker-config'), ('mountPath', '/kaniko/.docker')]), OrderedDict([('name', 'dockerfile-storage'), ('mountPath', '/data')])])])), ('restartPolicy', 'Never'), ('volumes', [OrderedDict([('name', 'docker-config'), ('secret', OrderedDict([('secretName', 'docker-config'), ('items', [OrderedDict([('key', '.dockerconfigjson'), ('path', 'config.json')])])]))]), OrderedDict([('name', 'dockerfile-storage'), ('persistentVolumeClaim', OrderedDict([('claimName', 'notebook-volume')]))])])])" is not None.
Error: Structure "OrderedDict([('container', OrderedDict([('image', 'gcr.io/kaniko-project/executor:debug'), ('command', ['busybox/sh', '-c']), ('args', ['/kaniko/executor', '--dockerfile=/data/Dockerfile', '--context=dir://data', '--destination=privaterepo/kanikotf:2.2.1', '--cache=true', '--skip-tls-verify']), ('volumeMounts', [OrderedDict([('name', 'docker-config'), ('mountPath', '/kaniko/.docker')]), OrderedDict([('name', 'dockerfile-storage'), ('mountPath', '/data')])])])), ('restartPolicy', 'Never'), ('volumes', [OrderedDict([('name', 'docker-config'), ('secret', OrderedDict([('secretName', 'docker-config'), ('items', [OrderedDict([('key', '.dockerconfigjson'), ('path', 'config.json')])])]))]), OrderedDict([('name', 'dockerfile-storage'), ('persistentVolumeClaim', OrderedDict([('claimName', 'notebook-volume')]))])])])" is incompatible with type "typing.Union[kfp.components._structures.ContainerImplementation, kfp.components._structures.GraphImplementation, NoneType]" - none of the types in Union are compatible.

 

Is there any way to pass volumes to Kaniko while building Docker Images.

Thanks