Unable to build using existing k3s installation (`address = "unix:///run/k3s/containerd/containerd.sock"` seems ignored)

[davidjade]

Description

I have an existing k3s installation and I recently set up nerdctl and buildkit and I cannot build images due to containerd socket issues.

This is the error I get:

error: failed to solve: failed to read dockerfile: failed to create lease: connection error: desc = "transport: error while dialing: dial unix:///run/containerd/containerd.sock: timeout": unavailable
FATA[0004] no image was built

contents of /etc/buildkit/buildkitd.toml

[worker.oci]
  enabled = false

[worker.containerd]
  enabled = true
  # namespace should be "k8s.io" for Kubernetes (including Rancher Desktop)
  namespace = "k8s.io"

contents of /etc/nerdctl/nerdctl.toml

debug          = false
debug_full     = false
address        = "unix:///run/k3s/containerd/containerd.sock"

I also tried using CONTAINERD_ADDRESS=/run/k3s/containerd/containerd.sock and it made no difference.

Note that if I do run a separate containerd system instance (and undo the above config), then I can build but not deploy (as the local images can't be found by k3s as they are in the other k3s containerd instance).

Also, if I symlink /run/containerd/containerd.sock to the k3s instance, then it works but I don't think I should have to do this.

Steps to reproduce the issue

1. Install k3s and ensure that it is functions correctly (test deployments, networking, etc...)
2. Install nerdctl-full-1.4.0-linux-arm64
3. Enable buildkit: systemctl enable --now buildkit
4. Setup /etc/buildkit/buildkitd.toml
5. Setup /etc/nerdctl/nerdctl.toml
6. nerdctl --namespace k8s.io build -t .

Describe the results you received and expected

That I could build successfully and then deploy on k3s using local images

What version of nerdctl are you using?

nerdctl-full-1.4.0-linux-arm64

Are you using a variant of nerdctl? (e.g., Rancher Desktop)

None

Host information

Client: Namespace: k8s.io Debug Mode: false

Server: Server Version: v1.6.19-k3s1 Storage Driver: overlayfs Logging Driver: json-file Cgroup Driver: systemd Cgroup Version: 2 Plugins: Log: fluentd journald json-file syslog Storage: native overlayfs fuse-overlayfs stargz Security Options: seccomp Profile: default cgroupns Kernel Version: 6.1.21-v8+ Operating System: Debian GNU/Linux 11 (bullseye) OSType: linux Architecture: aarch64 CPUs: 4 Total Memory: 7.629GiB Name: pibox ID: 2eb627de-37b8-4753-bc13-1068859b53c0