[FEATURE REQUEST] Allow pushing uncompressed images to local docker registry

[fruttasecca]

This is currently not possible given that the desired compression is hardcoded in vendor/github.com/containers/image/v5/docker/docker_image_dest.go, which leads to ignoring the --disable-compression flag on push.

func (d *dockerImageDestination) DesiredLayerCompression() types.LayerCompression {
	return types.Compress
}

FYI, I have tried to set it to types.PreserveOriginal and rebuilt buildah, which appears to work.

[flouthoc]

Hi @fruttasecca Thanks for opening the issue. This looks like a nice feature request, I'll try checking this.

[github-actions[bot]]

A friendly reminder that this issue had no activity for 30 days.

[rhatdan]

@mtrmac @vrothberg PTAL

[mtrmac]

Yes, this should be easily possible by making that an option in c/image/types.SystemContext, having dockerImageDestination.DesiredLayerCompression make a decision based on that option, and then hooking that option up to the Buildah/Podman CLI.

I’m afraid it’s not something I’m likely to work on any time soon. Care to prepare PRs?

[rhatdan]

As long as its possible, @flouthoc could you look into this?

[mtrmac]

Actually… @fruttasecca Why do you want this to be possible?

[flouthoc]

I'll take a look at this just waiting for @mtrmac 's concern to be answered.

[fruttasecca]

Hi @mtrmac, essentially we are in a situation of limited vcpus, images having a size of a few gigs (unbound since users are allowed to add more) and a registry that is deployed in the same cluster as the buildah pod. We favoured speed over space, pushing with types.PreserveOriginal allowed us to reduce the total build+push time by almost 50%.

[mtrmac]

Thanks, that makes sense.

(Often enough, these flag requests are motivated, after digging 3-4 steps deep, by some completely unrelated bug or missing feature, so I think it’s always worth asking.)

[fruttasecca]

Makes sense, thank you for looking into it :+1: .

[rhatdan]

@flouthoc Looks like we are a go.

[github-actions[bot]]

A friendly reminder that this issue had no activity for 30 days.

[ricklamers]

We'd be happy to contribute to help get this feature over the line. Anything in particular the core team is still thinking about in relation to this being added?

We're now maintaining a patched fork which is a bit cumbersome with multi-arch CI.

Thanks for building an awesome tool 🙏

[flouthoc]

@ricklamers Sure, would you like to share the patch or create PR if possible ? Otherwise i'll try to get this in coming days.

[flouthoc]

@ricklamers @fruttasecca Could you also help me know how are you testing this ? I see no difference in the size of pushed blobs/layers i am testing this by pusing to dockerhub ( tried both --format docker and --format oci )

func (d *dockerImageDestination) DesiredLayerCompression() types.LayerCompression {
-	return types.Compress
+	return types.PreserveOriginal
} 

if you have a patch in handy could you please share it.

[fruttasecca]

Hi @flouthoc, thanks for looking into it. We are building buildah 1.26 from this branch https://github.com/orchest/buildah/tree/experimental/1.26.0-no-docker-registry-compression. This (https://github.com/containers/buildah/commit/fe6b5ad0ef60af65f82306b8a580e32003eb1ade) is the only change we needed to do, and this is how we build the image https://github.com/orchest/buildah/blob/experimental/1.26.0-no-docker-registry-compression/Dockerfile.

This is how we are running the CLI in a pod, pushing to a docker registry deployed in a k8s cluster, not dockerhub (this is an interpolated & multiline python string)

# Build
f"buildah build -f {dockerfile_path} --layers=true "
# Package managers caches.
# jovyan is the user of the base image.
"-v /pip-cache:/home/jovyan/.cache/pip "
# Obtained by running "conda info". Note
# that this cache is also used by mamba.
"-v /conda-cache:/opt/conda/pkgs "
# https://github.com/containers/buildah/issues/2741
"--format docker "
"--force-rm=true "
"--disable-compression=true "
# Avoid a warning about not being able
# to write to the audit log.
"--cap-add=CAP_AUDIT_WRITE "
f"--tag {full_image_name} "
# Push
"&& buildah push "
"--disable-compression=true "
# Buildah might compress regardless of
# the specified options depending on the
# destination storage, tune such
# compression.
"--compression-format=zstd:chunked "
"--compression-level=0 "
f"{full_image_name}"

About disabling compression, it's not a hard requirement but more like something that we had to do to reduce cpu overhead in a context of limited resources. Happy to go through some more benchmarking and playing around with --flags in the future to see where/if the sweet spot has moved with new buildah releases.

[mtrmac]

I see no difference in the size of pushed blobs/layers i am testing this by pusing to dockerhub

Are the blobs actually being pushed? See the debug log; you might need to delete the blob info cache, otherwise the push code will know that there is already a version on the registry, and prefer that one regardless of DesiredLayerCompression.

[github-actions[bot]]

A friendly reminder that this issue had no activity for 30 days.

[rhatdan]

Since you never answered @mtrmac I am going to assume you have this working. Closing. Reopen if you have more feedback or this issue is ongoing.

[flouthoc]

I see no difference in the size of pushed blobs/layers i am testing this by pusing to dockerhub

Are the blobs actually being pushed? See the debug log; you might need to delete the blob info cache, otherwise the push code will know that there is already a version on the registry, and prefer that one regardless of DesiredLayerCompression.

Whoops this was a question for me and not for the issue author so issue is still not resolved, just re-opening. :)

[github-actions[bot]]

A friendly reminder that this issue had no activity for 30 days.

[rhatdan]

@flouthoc any progress?

[github-actions[bot]]

A friendly reminder that this issue had no activity for 30 days.

[rhatdan]

@flouthoc any progress?

[github-actions[bot]]

A friendly reminder that this issue had no activity for 30 days.

[ryeyao]

Is there anyone still working on this?

[flouthoc]

@ryeyao No but I'll take a look thanks for the poke.

[alexdmiller]

Anyone working on this?

[rhatdan]

Doubt it. @flouthoc is off working on his graduate degree, so the Buildah project could use more help.