storage icon indicating copy to clipboard operation
storage copied to clipboard

Podman slow to push images when using zstd:chunked

Open noelmiller opened this issue 1 year ago • 0 comments

Problem

Pushing to a registry takes far longer using ZSTD chunked than using traditional gzip compression. We believe the reason for this is because it needs to do compression on each of the many layers in an OS Image. This issue compounds itself if you are pushing multiple tags rather than just a single tag. The upload of a gzip image takes about 3 - 10 minutes with 5 tags. A ZSTD chunked image takes about 35 minutes with 5 tags. If there is a failure in the push at any point, you will need to push it again. This results in far longer build times for us.

Examples

Here is a recent push for Bazzite using ZSTD chunked: https://github.com/ublue-os/bazzite/actions/runs/11604220311/job/32315410778#step:28:1

Here is a previous build using gzip: https://github.com/ublue-os/bazzite/actions/runs/11332889740/job/31515959369#step:28:1

noelmiller avatar Nov 01 '24 00:11 noelmiller