no space left on device when `vctl pull`

[zcqian]

When pulling very large containers, it may be possible that vctl pull runs out of space when applying the layers/extracting images.

For example, I can recreate the issue on my setup by running vctl pull readthedocs/build:stable.

What determines the temporary image size? I can't find any options in the man page.

[yeahdongcn]

There are too many layers AFAIK (https://hub.docker.com/layers/readthedocs/build/master/images/sha256-f9292ccc6fda59fcc45df925e7c4068896b5f5bc9fc4ec1f2b9be39aad5553e7?context=explore), vctl actually uses a snapshotter which is similar to the native one, so each snapshot contains whole contents in the previous snapshot. That's the reason why pulling readthedocs/build:stable out of space.

[zcqian]

According to my observation, it's not because the Fusion Container Storage.sparseimage running out of space but individual snapshots. It's true that they contain everything, but I think the issue is that each new snapshot(can be found under ~/.vctl/storage/containerd/root/io.containerd.snapshotter.v1.dmg/snapshots) is capped by the same size limits of ~4GB.

[zcqian]

I tried manually suspending vctl and growing the first snapshot image, it got "further" and the image eventually grew beyond 4GB. Then after a few more layers, Fusion Container Storage.sparseimage became full and that's where each snapshot contains all previous layers + having too many layers starts to become a problem (a 128GB image can't fit too many 5GB images inside).

I can definitely work around these issues by manually resizing all the images, but this certainly isn't good user experience. I will leave the discussion here so anyone else running into the issue can at least work around it by resizing the sparse images.