[BUG] k3d image import does not propagate DOCKER_SOCK to tools node/container

[aescaler-raft]

What did you do

• How was the cluster created?

  • export DOCKER_HOST=ssh://root@localhost:55555 DOCKER_SOCK=/run/podman/podman.sock
  • podman network create mycluster
  • k3d registry create --default-network mycluster mycluster-registry
  • k3d cluster create --verbose --network mycluster --registry-use mycluster-registry:55556 mycluster

• What did you do afterwards?

  • k3d image import localhost/raft/testapp:v0.0.0-arm64 -c mycluster --verbose -t

What did you expect to happen

k3d imports image into cluster using tools node/container

Screenshots or terminal output

Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?

Which OS & Architecture

arch: arm64
cgroupdriver: systemd
cgroupversion: "2"
endpoint: /run/podman/podman.sock
filesystem: xfs
infoname: localhost.localdomain
name: docker
os: fedora
ostype: linux
version: 4.5.0

Which version of k3d

k3d version v5.4.9
k3s version v1.25.7-k3s1 (default)

Which version of podman

Client:       Podman Engine
Version:      4.5.0
API Version:  4.5.0
Go Version:   go1.20.3
Git Commit:   75e3c12579d391b81d871fd1cded6cf0d043550a
Built:        Fri Apr 14 09:28:20 2023
OS/Arch:      darwin/arm64

Server:       Podman Engine
Version:      4.5.0
API Version:  4.5.0
Go Version:   go1.20.2
Built:        Fri Apr 14 11:41:12 2023
OS/Arch:      linux/arm64

host:
  arch: arm64
  buildahVersion: 1.30.0
  cgroupControllers:
  - cpu
  - io
  - memory
  - pids
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-2.1.7-2.fc38.aarch64
    path: /usr/bin/conmon
    version: 'conmon version 2.1.7, commit: '
  cpuUtilization:
    idlePercent: 99.79
    systemPercent: 0.11
    userPercent: 0.1
  cpus: 1
  databaseBackend: boltdb
  distribution:
    distribution: fedora
    variant: coreos
    version: "38"
  eventLogger: journald
  hostname: localhost.localdomain
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 1000000
    uidmap:
    - container_id: 0
      host_id: 501
      size: 1
    - container_id: 1
      host_id: 100000
      size: 1000000
  kernel: 6.2.13-300.fc38.aarch64
  linkmode: dynamic
  logDriver: journald
  memFree: 1535688704
  memTotal: 2049077248
  networkBackend: netavark
  ociRuntime:
    name: crun
    package: crun-1.8.4-1.fc38.aarch64
    path: /usr/bin/crun
    version: |-
      crun version 1.8.4
      commit: 5a8fa99a5e41facba2eda4af12fa26313918805b
      rundir: /run/user/501/crun
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +LIBKRUN +WASM:wasmedge +YAJL
  os: linux
  remoteSocket:
    exists: true
    path: /run/user/501/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: true
  serviceIsRemote: true
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.2.0-12.fc38.aarch64
    version: |-
      slirp4netns version 1.2.0
      commit: 656041d45cfca7a4176f6b7eed9e4fe6c11e8383
      libslirp: 4.7.0
      SLIRP_CONFIG_VERSION_MAX: 4
      libseccomp: 2.5.3
  swapFree: 0
  swapTotal: 0
  uptime: 2h 24m 43.00s (Approximately 0.08 days)
plugins:
  authorization: null
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  - ipvlan
  volume:
  - local
registries:
  localhost:55556:
    Blocked: false
    Insecure: true
    Location: localhost:55556
    MirrorByDigestOnly: false
    Mirrors: null
    Prefix: localhost:55556
    PullFromMirror: ""
  search:
  - docker.io
store:
  configFile: /var/home/core/.config/containers/storage.conf
  containerStore:
    number: 11
    paused: 0
    running: 0
    stopped: 11
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /var/home/core/.local/share/containers/storage
  graphRootAllocated: 106769133568
  graphRootUsed: 8630263808
  graphStatus:
    Backing Filesystem: xfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Using metacopy: "false"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 59
  runRoot: /run/user/501/containers
  transientStore: false
  volumePath: /var/home/core/.local/share/containers/storage/volumes
version:
  APIVersion: 4.5.0
  Built: 1681486872
  BuiltTime: Fri Apr 14 11:41:12 2023
  GitCommit: ""
  GoVersion: go1.20.2
  Os: linux
  OsArch: linux/arm64
  Version: 4.5.0

[aescaler-raft]

relevant output from podman container inspect k3d-mycluster-tools

Mounts:

               {
                    "Type": "bind",
                    "Source": "/run/podman/podman.sock",
                    "Destination": "/run/podman/podman.sock",
                    "Driver": "",
                    "Mode": "",
                    "Options": [
                         "nosuid",
                         "nodev",
                         "rbind"
                    ],
                    "RW": true,
                    "Propagation": "rprivate"
               }

Env:

               "Env": [
                    "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                    "TERM=xterm",
                    "container=podman",
                    "K3S_KUBECONFIG_OUTPUT=/output/kubeconfig.yaml",
                    "HOSTNAME=k3d-bigbang-tools",
                    "HOME=/root"
               ],

[aescaler-raft]

Workaround

A usable workaround is to deploy the registry with k3d registry create and push to that registry. The cluster is then able to pull from that registry successfully.

[aescaler-raft]

k3d/pkg/client/tools.go seems to be the relevant file.