containerd integration: `docker system df` reports negative `Reclaimable` for images

[vvoland]

Description

Reaclaimable column from docker system df shows negative (and wrong!) size.

Reproduce

$ docker system df
TYPE                TOTAL               ACTIVE              SIZE                RECLAIMABLE
Images              0                   0                   0B                  0B
Containers          0                   0                   0B                  0B
Local Volumes       0                   0                   0B                  0B
$ docker pull ubuntu
Using default tag: latest
67211c14fa74: Download complete
537da2481863: Download complete
bab8ce5c00ca: Download complete
cd741b12a7ea: Download complete
$ docker system df
TYPE                TOTAL               ACTIVE              SIZE                RECLAIMABLE
Images              1                   0                   78.97MB             -2.735e+07B (-34%)
Containers          0                   0                   0B                  0B
Local Volumes       0                   0                   0B                  0B

Expected behavior

Don't make user think that space will be lost after deleting an image 😄

docker version

master - a343ed13e556ec97a26bd5fc34b069a64f86f16b

docker info

-

Additional Info

No response

[vvoland]

Looks like LayerDiskUsage should also include image content size:

https://github.com/moby/moby/blob/a343ed13e556ec97a26bd5fc34b069a64f86f16b/daemon/containerd/service.go#L124-L137

[Anutrix]

Facing same issue in Docker Desktop 4.21.1 (114176).

[thaJeztah]

@Anutrix is that with the containerd image store feature enabled, or without?

[ctalledo]

Hi @vvoland, @thaJeztah, I can't reproduce this with Docker 28.0.1, either with the containerd image store or with the docker image store.

$ docker system df
TYPE            TOTAL     ACTIVE    SIZE      RECLAIMABLE 
Images          0         0         0B        0B
Containers      0         0         0B        0B
Local Volumes   0         0         0B        0B            
Build Cache     0         0         0B        0B

$ docker pull ubuntu
Using default tag: latest                   
latest: Pulling from library/ubuntu
5a7813e071bf: Pull complete 
Digest: sha256:72297848456d5d37d1262630108ab308d3e9ec7ed1c3286a32fe09856619a782
Status: Downloaded newer image for ubuntu:latest
docker.io/library/ubuntu:latest

$ docker system df                                                                             
TYPE            TOTAL     ACTIVE    SIZE      RECLAIMABLE
Images          1         0         78.13MB   78.13MB (100%)
Containers      0         0         0B        0B
Local Volumes   0         0         0B        0B
Build Cache     0         0         0B        0B

Can you try reproducing? If it doesn't repro we should close this.

[jsternberg]

Trying to identify why the original reproducer worked, but this is still reproducible if you do something to cause the image to be unreclaimable.

$ docker pull ubuntu
$ docker create ubuntu
$ docker system df
TYPE            TOTAL     ACTIVE    SIZE      RECLAIMABLE
Images          1         1         110.4MB   -2.887e+07B (-26%)
Containers      1         0         4.096kB   4.096kB (100%)
Local Volumes   0         0         0B        0B
Build Cache     0         0         0B        0B

If you look at the JSON return, the LayerSize for images is less than the Size of the image which causes an incorrect calculation in the CLI: https://github.com/docker/cli/blob/f1385df2a7213cb38258c845d18ff5d6a88ef535/cli/command/formatter/disk_usage.go#L296-L313.

{
  "LayersSize": 110407680,
  "Images": [
    {
      "Containers": 1,
      "Created": 1737951294,
      "Id": "sha256:72297848456d5d37d1262630108ab308d3e9ec7ed1c3286a32fe09856619a782",
      "Labels": {
        "org.opencontainers.image.ref.name": "ubuntu",
        "org.opencontainers.image.version": "24.04"
      },
      "ParentId": "",
      "Descriptor": {
        "mediaType": "application/vnd.oci.image.index.v1+json",
        "digest": "sha256:72297848456d5d37d1262630108ab308d3e9ec7ed1c3286a32fe09856619a782",
        "size": 6688
      },
      "RepoDigests": [
        "ubuntu@sha256:72297848456d5d37d1262630108ab308d3e9ec7ed1c3286a32fe09856619a782"
      ],
      "RepoTags": [
        "ubuntu:latest"
      ],
      "SharedSize": 0,
      "Size": 139279440
    }
  ],
  "Containers": [
    {
      "Id": "38ee2d8946325ba7369e6474d59c15e25f9ffaf61e31ebcf81ce48139e27583f",
      "Names": [
        "/naughty_kepler"
      ],
      "Image": "ubuntu",
      "ImageID": "sha256:72297848456d5d37d1262630108ab308d3e9ec7ed1c3286a32fe09856619a782",
      "Command": "/bin/bash",
      "Created": 1743007505,
      "Ports": [],
      "SizeRw": 4096,
      "SizeRootFs": 110407680,
      "Labels": {
        "org.opencontainers.image.ref.name": "ubuntu",
        "org.opencontainers.image.version": "24.04"
      },
      "State": "created",
      "Status": "Created",
      "HostConfig": {
        "NetworkMode": "bridge"
      },
      "NetworkSettings": {
        "Networks": {
          "bridge": {
            "IPAMConfig": null,
            "Links": null,
            "Aliases": null,
            "MacAddress": "",
            "DriverOpts": null,
            "GwPriority": 0,
            "NetworkID": "",
            "EndpointID": "",
            "Gateway": "",
            "IPAddress": "",
            "IPPrefixLen": 0,
            "IPv6Gateway": "",
            "GlobalIPv6Address": "",
            "GlobalIPv6PrefixLen": 0,
            "DNSNames": null
          }
        }
      },
      "Mounts": []
    }
  ],
  "Volumes": [],
  "BuildCache": []
}

[vvoland]

Yes, for containerd we should also include the blob content size in: https://github.com/moby/moby/issues/45395#issuecomment-1521893812

(and so the LayerDiskUsage should probably be renamed to ImagesDiskUsage?)