dcrd icon indicating copy to clipboard operation
dcrd copied to clipboard

docker: Build and push release image

Open peterzen opened this issue 2 years ago • 9 comments

This PR adds a Github workflow that executes when a new release is created on GH. It runs a multi-platform docker build (amd64 and arm64), logs into Docker Hub using credentials configured in org-wide secrets and finally pushes the images to the official Decred Docker Hub repository.

The workflow is similar to the one deployed in decred/dcrdex: https://github.com/decred/dcrdex/blob/master/.github/workflows/release-docker.yml

peterzen avatar Feb 25 '23 23:02 peterzen

Thanks for the PR.

I'm not entirely sure we really want to do this. The problem is that if we add this, we will essentially be taking on distributing official docker images, which we have, rather intentionally, not done to date due to security implications.

Perhaps more importantly is the fact that a release via this method is not the same as the reproducible builds that we have multiple people verify and sign.

Since @jrick wrote the reproducible release tools we use, I'd like to get his perspective here.

davecgh avatar Feb 26 '23 17:02 davecgh

Just to give a bit of background/motivation for this PR - a Docker image (ideally officially published) is a prerequisite for integration on platforms like Umbrel and Start9, which will hopefully be helpful in improving the running node count.

peterzen avatar Feb 27 '23 19:02 peterzen

I'm not totally sure what this is doing but I don't think this is creating images containing our official release binaries? It doesn't make sense to me that we go through the effort to verify and test those builds, but then build a docker image for the release containing something else.

jrick avatar Feb 27 '23 19:02 jrick

Would an alternative implementation of this workflow that builds the Docker image from the signed released binaries be acceptable?

peterzen avatar Feb 27 '23 19:02 peterzen

I'm not totally sure what this is doing but I don't think this is creating images containing our official release binaries? It doesn't make sense to me that we go through the effort to verify and test those builds, but then build a docker image for the release containing something else.

Correct. It builds them from source (although it does use the same flags, so it's probably the same up to that point, at least), but then it also does upx to reduce the size of the binaries quite a bit. More importantly though, they clearly aren't signed as the real releases are either.

Speaking of upx compression, we might actually want to start doing as part of the release build if it can do it cross platform, which I think it can.

The following shows the sizes of the binaries in the image with and without upx:

Binary Without upx With upx
dcrctl 6.45M 3.1M
dcrd 11.9M 5.2M
entrypoint 1.8M 0.7M
gencerts 3.2M 1.3M

davecgh avatar Feb 27 '23 19:02 davecgh

Would an alternative implementation of this workflow that builds the Docker image from the signed released binaries be acceptable?

I think that is probably the best option if we're going to do official docker images. It really needs to be based off the release bins imo. I mentioned that a bit in the other PR that allowed different branches/tags to be built (which is useful its own right, so not a waste).

Quoting from there for convenience:

I wonder if it would make more sense instead for a Dockerfile.release to download the specific release binaries, import the gpg public release keys, and verify the signatures, instead of building from source.

davecgh avatar Feb 27 '23 20:02 davecgh

I should also note that if we go that approach, we can't tie it to a release tag in the dcrd repository either.

EDIT: I mean we can't have it automatically build and push the image(s) on the push of a release tag.

davecgh avatar Feb 27 '23 20:02 davecgh

I should also note that if we go that approach, we can't tie it to a release tag in the dcrd repository either.

EDIT: I mean we can't have it automatically build the image on the push of a release tag.

Indeed, the docker build would then have to be added to the end of the (manual?) process that produces the signed binaries.

peterzen avatar Feb 27 '23 20:02 peterzen

the signatures are manually generated, but are uploaded to a github draft release prior to publishing. by the time that release/tag is created, the sigs can be used.

the best places to run something like this would be the decred-binaries or decred-release repo. probably decred-binaries, since it has the binaries for everything except dcrinstall. but some of the releases published there are of e.g. decrediton only, where dcrd/dcrwallet/etc. doesn't get any update.

jrick avatar Feb 27 '23 20:02 jrick