Question on availability of new plugin packages

[miknight]

Hi folks,

I appreciate there's likely some considerations here that aren't visible to the casual observer but I was hoping to get some clarity on the availability of packages published to https://download.docker.com for new plugin releases (e.g. https://github.com/docker/buildx/releases/tag/v0.10.4).

For example, I notice that this repo has merged a PR relating to the buildx v0.10.4 release (https://github.com/docker/docker-ce-packaging/pull/853) a couple of weeks ago but there's been nothing new in (for example) https://download.docker.com/linux/ubuntu/dists/jammy/pool/stable/amd64/ for over a month. Is there some kind of blocker we could keep an eye on to stay informed on the progress?

Thanks for any insights!

[ericslandry]

Related, I saw that:

• Docker Compose releases a new version: https://github.com/docker/compose/releases/tag/v2.17.1
• This repo updates a variable via PR: https://github.com/docker/docker-ce-packaging/pull/863

But I don't know which process publishes the docker-compose-plugin deb package to the Docker APT repository?

I would also be grateful for any insights!

[thaJeztah]

Releases of plugins (more factually: publishing of .deb and .rpm packages for those on download.docker.com) are currently coupled with releases of the Docker Engine and CLI. This is a limitation of our current release-pipeline, which was created when "docker" was a single package, and the only "release vehicle" was the Docker Engine release. Docker being a single package is no longer the case (e.g., we now have separate packages for the docker CLI, the Docker Engine (dockerd), as containerd, compose, buildx, etc).

So the current flow is that;

• a component does a release (e.g. https://github.com/docker/buildx/releases/tag/v0.10.5)
• component versions are updated in this repository in the actively maintained release-branches (https://github.com/docker/docker-ce-packaging/pull/900, https://github.com/docker/docker-ce-packaging/pull/901)
• updated packages are built and released together with the next (patch) release of the Docker Engine (e.g. in this case buildx .deb and .rpm will be published with either Docker Engine v24.0.2 or v23.0.7, whichever comes first)
• Docker Desktop releases may also bundle CLI plugins, and plugin versions in those releases are updated with each release of Docker Desktop (if new versions are available).

I should mention that (most / all of) the CLI plugins are static binaries, so if you need access to a release that's not yet published on download.docker.com, you can manually download a binary from their respective GitHub repository, and install them in one of the plugin directories that the CLI looks for. Keep in mind that (depending on where you install) may take precedence over the locations in which the .deb and .rpm packages are installed, so should be removed once you installed updated .deb or .rpm packages.

Where possible we try to publish packages of those components as soon as possible after they did a release, but sometimes there may be some latency if they "just" missed the release train. This is something we're actively working on, but involves various parts in our pipeline (as well as splitting our static packages into separate ones), and still takes some time to complete. Part of this work is done in https://github.com/docker/packaging, which is a rewrite of various repositories, but that's only the "building" part (publishing, signing, verifying packages is currently an internal pipeline).