Microk8s: unable to start built container

[MarcusAhlfors]

What steps did you take and what happened

• Successfully installed buildkit
• Tried to do the same exact steps as in the example on a microk8s single node cluster
• The build did run successfully
• After the build I couldn't start the pod
• The built image was not found in containerd image list

What did you expect to happen

Container would be uploaded to containerd and the pod would start

Environment Details:

• kubectl buildkit v0.1.0
• Kubernetes v1.19.3-34+a56971609ff35a
• MicroK8s on ubuntu 20.04 (< 1 month old version)
• containerd v1.3.7

Details

MicroK8s has containerd socket at: /var/snap/microk8s/common/run/containerd.sock, i tried doing:

kubectl buildkit create --runtime containerd --containerd-sock=/var/snap/microk8s/common/run/containerd.sock

But then not even the build worked anymore.

I also tried symlinking a few locations back and forth but none of these helped

ln -s /var/snap/microk8s/common/run/containerd.sock /var/run/containerd.sock
ln -s /var/snap/microk8s/common/var/lib/containerd /var/lib/containerd
ln -s /var/snap/microk8s/common/run/containerd /run/containerd

[MarcusAhlfors]

Similar to https://github.com/vmware-tanzu/buildkit-cli-for-kubectl/issues/46#issuecomment-729873677_ but specific to MicroK8s

[pdevine]

@MarcusAhlfors what was the command you used to build?

[MarcusAhlfors]

Initially I tried to do the exact same as in your example svg:

kubectl build -t pants:kube -f Dockerfile ./

Which looked like it worked but the image never showed up in containerd, after that I tried lot's of things. I'm happy to help I there's anything I could do, I really like what you have done and it could be useful to me in a project I'm working on.

[pdevine]

@MarcusAhlfors I installed microk8s on ubuntu 20.04 and was able to reproduce the issue. I also have docker installed, so when the build completed it actually loaded the image into my docker runtime by default and I was able to run it with docker run .... Do you also have docker installed?

[pdevine]

@MarcusAhlfors there is a workaround if you have docker installed, which is that you can use the local microk8s registry with something like:

$ microk8s enable registry
$ microk8s kubectl build -t localhost:32000/pants:kube ./
$ docker push localhost:32000/pants:kube
$ microk8s kubectl run -it --rm --image localhost:32000/pants:kube pants

I've been digging around inside of ctr with namespaces trying to figure out what's going on, but microk8s does some pretty funky stuff. Will need to keep digging.

[tomikonio]

I am using kind and have exactly the same issue:
The kubectl build command runs successfully.
The image does not appear in the kind cluster.

[dhiltgen]

I am using kind and have exactly the same issue: The kubectl build command runs successfully. The image does not appear in the kind cluster.

@tomikonio can you take a look at #79 and see if that might be what's going on in your environment? If not, please file a new issue to track the specifics of your kind failure so we can investigate what's going on.