kfp_samples
kfp_samples copied to clipboard
Attaching Volume to Kaniko When Building Docker Images using Kaniko In Kubeflow Pipeline
Hi @Ark-kun , First of all thanks for the sample Notebooks.
I am trying to build Docker Image using Kaniko in my Kubeflow Pipeline as my environment doesn't have Internet access also I need to push Docker Image to private repository so that I can use the image in my next pipeline step.
kfp Version: 0.1.32 Component.yaml
name: Build Image Using Kaniko
description: Stage is for building Docker Image Using Kanio.
implementation:
container:
image: gcr.io/kaniko-project/executor:debug
command: ["busybox/sh", "-c"]
args: [
"/kaniko/executor",
--dockerfile=/data/Dockerfile,
--context=dir://data,
--destination=privaterepo/kanikotf:2.2.1,
--cache=true,
--skip-tls-verify
]
volumeMounts:
- name: docker-config
mountPath: /kaniko/.docker
- name: dockerfile-storage
mountPath: /data
restartPolicy: Never
volumes:
- name: docker-config
secret:
secretName: docker-config
items:
- key: .dockerconfigjson
path: config.json
- name: dockerfile-storage
persistentVolumeClaim:
claimName: notebook-volume
While loading Yaml:
build_image_op = kfp.components.load_component_from_text(kaniko_yaml)
Error:
TypeError: Error: ContainerImplementation.from_dict(struct=OrderedDict([('container', OrderedDict([('image', 'gcr.io/kaniko-project/executor:debug'), ('command', ['busybox/sh', '-c']), ('args', ['/kaniko/executor', '--dockerfile=/data/Dockerfile', '--context=dir://data', '--destination=privaterepo/kanikotf:2.2.1', '--cache=true', '--skip-tls-verify']), ('volumeMounts', [OrderedDict([('name', 'docker-config'), ('mountPath', '/kaniko/.docker')]), OrderedDict([('name', 'dockerfile-storage'), ('mountPath', '/data')])])])), ('restartPolicy', 'Never'), ('volumes', [OrderedDict([('name', 'docker-config'), ('secret', OrderedDict([('secretName', 'docker-config'), ('items', [OrderedDict([('key', '.dockerconfigjson'), ('path', 'config.json')])])]))]), OrderedDict([('name', 'dockerfile-storage'), ('persistentVolumeClaim', OrderedDict([('claimName', 'notebook-volume')]))])])])) failed with exception:
Error: ContainerSpec.from_dict(struct=OrderedDict([('image', 'gcr.io/kaniko-project/executor:debug'), ('command', ['busybox/sh', '-c']), ('args', ['/kaniko/executor', '--dockerfile=/data/Dockerfile', '--context=dir://data', '--destination=privaterepo/kanikotf:2.2.1', '--cache=true', '--skip-tls-verify']), ('volumeMounts', [OrderedDict([('name', 'docker-config'), ('mountPath', '/kaniko/.docker')]), OrderedDict([('name', 'dockerfile-storage'), ('mountPath', '/data')])])])) failed with exception:
__init__() got an unexpected keyword argument 'volumeMounts'
Error: GraphImplementation.from_dict(struct=OrderedDict([('container', OrderedDict([('image', 'gcr.io/kaniko-project/executor:debug'), ('command', ['busybox/sh', '-c']), ('args', ['/kaniko/executor', '--dockerfile=/data/Dockerfile', '--context=dir://data', '--destination=privaterepo/kanikotf:2.2.1', '--cache=true', '--skip-tls-verify']), ('volumeMounts', [OrderedDict([('name', 'docker-config'), ('mountPath', '/kaniko/.docker')]), OrderedDict([('name', 'dockerfile-storage'), ('mountPath', '/data')])])])), ('restartPolicy', 'Never'), ('volumes', [OrderedDict([('name', 'docker-config'), ('secret', OrderedDict([('secretName', 'docker-config'), ('items', [OrderedDict([('key', '.dockerconfigjson'), ('path', 'config.json')])])]))]), OrderedDict([('name', 'dockerfile-storage'), ('persistentVolumeClaim', OrderedDict([('claimName', 'notebook-volume')]))])])])) failed with exception:
__init__() got an unexpected keyword argument 'container'
Error: Structure "OrderedDict([('container', OrderedDict([('image', 'gcr.io/kaniko-project/executor:debug'), ('command', ['busybox/sh', '-c']), ('args', ['/kaniko/executor', '--dockerfile=/data/Dockerfile', '--context=dir://data', '--destination=privaterepo/kanikotf:2.2.1', '--cache=true', '--skip-tls-verify']), ('volumeMounts', [OrderedDict([('name', 'docker-config'), ('mountPath', '/kaniko/.docker')]), OrderedDict([('name', 'dockerfile-storage'), ('mountPath', '/data')])])])), ('restartPolicy', 'Never'), ('volumes', [OrderedDict([('name', 'docker-config'), ('secret', OrderedDict([('secretName', 'docker-config'), ('items', [OrderedDict([('key', '.dockerconfigjson'), ('path', 'config.json')])])]))]), OrderedDict([('name', 'dockerfile-storage'), ('persistentVolumeClaim', OrderedDict([('claimName', 'notebook-volume')]))])])])" is not None.
Error: Structure "OrderedDict([('container', OrderedDict([('image', 'gcr.io/kaniko-project/executor:debug'), ('command', ['busybox/sh', '-c']), ('args', ['/kaniko/executor', '--dockerfile=/data/Dockerfile', '--context=dir://data', '--destination=privaterepo/kanikotf:2.2.1', '--cache=true', '--skip-tls-verify']), ('volumeMounts', [OrderedDict([('name', 'docker-config'), ('mountPath', '/kaniko/.docker')]), OrderedDict([('name', 'dockerfile-storage'), ('mountPath', '/data')])])])), ('restartPolicy', 'Never'), ('volumes', [OrderedDict([('name', 'docker-config'), ('secret', OrderedDict([('secretName', 'docker-config'), ('items', [OrderedDict([('key', '.dockerconfigjson'), ('path', 'config.json')])])]))]), OrderedDict([('name', 'dockerfile-storage'), ('persistentVolumeClaim', OrderedDict([('claimName', 'notebook-volume')]))])])])" is incompatible with type "typing.Union[kfp.components._structures.ContainerImplementation, kfp.components._structures.GraphImplementation, NoneType]" - none of the types in Union are compatible.
Is there any way to pass volumes to Kaniko while building Docker Images.
Thanks