Cant remove cache from pods in external kubernetes context

[andrey-gava]

I don't know is it a bug or more a question. Cant find an answer to this anywhere.

So I created external kubernetes context docker buildx create --name portal-cluster --driver kubernetes --driver-opt replicas=4,namespace=docker-buildx,loadbalance=sticky docker buildx use portal-cluster Build test image docker buildx build -t nexus.domain.com:445/portal-notary:434 -o type=registry --push --cache-to=type=inline --progress=plain . Within builder pod I can see that cache created

/var/lib/buildkit # du -sh *
0	buildkitd.lock
52.0K	cache.db
821.9M	runc-overlayfs

If I run buildx build again, as expected it will use some layers from cache. But I need in some cases prune cache if disk usage become to large. I tried execute

docker buildx prune --all
docker buildx prune --verbose --builder portal-cluster
docker builder prune --all

But used space inside pod still the same and at build time layers still cached. Only by terminate pod I can delete cache inside pod. docker buildx du return

Reclaimable:	0B
Total:		0B

[andrey-gava]

docker version

Client: Docker Engine - Community
 Version:           20.10.11
 API version:       1.41
 Go version:        go1.16.9
 Git commit:        dea9396
 Built:             Thu Nov 18 00:38:53 2021
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.11
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.16.9
  Git commit:       847da18
  Built:            Thu Nov 18 00:37:17 2021
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.4.12
  GitCommit:        7b11cfaabd73bb80907dd23182b9347b4245eb5d
 runc:
  Version:          1.0.2
  GitCommit:        v1.0.2-0-g52b36a2
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Kubernetes Server Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.20", GitCommit:"1f3e19b7beb1cc0110255668c4238ed63dadb7ad", GitTreeState:"clean", BuildDate:"2021-06-16T12:51:17Z", GoVersion:"go1.13.15", Compiler:"gc", Platform:"linux/amd64"}

[tonistiigi]

@morlay ideas?