Checksums for released Windows installers

[joeapearson]

Is your feature request related to a problem? Please describe.

We want to ensure the integrity of windows installer downloads. We're investigating creation of a windows container base image and it's common practice to perform a checksum of externally downloaded artifacts when building an image.

(if the checksums are available somewhere and I'm just missing them please let me know, I've had a hunt on the website and on the github repo)

Describe the solution you'd like

Add checksums for the windows installers to (for example):

• https://github.com/erlang/otp/releases/download/OTP-24.0.4/SHA256.txt
• https://github.com/erlang/otp/releases/download/OTP-24.0.4/MD5.txt

Describe alternatives you've considered

• Generating our own checksums and then verifying them; but we can hardly consider that "official".

[garazdawi]

The checksums are available here: http://erlang.org/download/MD5

It should be relatively easy to add a step to our github actions main.yaml and sync-github-releases.yaml workflows that also include the checksum in the github release. Not sure when/if we will work on this. As always a PR would help speed things along.

[joeapearson]

Thanks @garazdawi for the MD5 sums and the hint about where the changes should be made. I'll look into a PR.

[rgl]

Friendly reminder that this is still not appearing in the latest 25.0 release.

[garazdawi]

@rgl Friendly reminder that a PR is still welcome!

[rgl]

I tried to understand the build process, but I'm kinda lost.

This is what I was able to understand:

1. a tag is created in this erlang/otp repo, e.g., at erlang/otp/releases/tag/OTP-25.0.
2. the github actions main.yaml workflow is executed.
   1. while it executes, it creates a bunch of artifacts, e.g., at https://github.com/erlang/otp/actions/runs/2345384305:
      • among them, there's the otp_win32_installer, which contains:
        • otp_win64_25.0.exe
   2. at some point, it creates the following files and uploads them to the github release, e.g., at:
      • MD5.txt (only for the following .tar.gz files)
      • SHA256.txt (only for the following .tar.gz files)
      • otp_doc_html_25.0.tar.gz
      • otp_doc_man_25.0.tar.gz
      • otp_src_25.0.tar.gz
   3. then, it triggers the remote erlang/erlang-org update-gh-cache.yaml workflow execution, e.g., at:
      • https://github.com/erlang/erlang-org/actions/workflows/update-gh-cache.yaml
      • https://github.com/erlang/erlang-org/actions/runs/2345495137

Then, periodically, the github actions sync-github-releases.yaml workflow is executed, which does:

1. executes .github/scripts/sync-github-releases.sh
   1. downloads binaries from erlang.org
      • I'm lost here. What exactly is uploading these binaries to erlang.org? The main.yaml workflow nor the erlang/erlang-org update-gh-cache.yaml workflow seem to do that.
   2. uploads the binaries into the existing github release.

But, what is actually uploading the main.yaml workflow artifacts to erlang.org?

[garazdawi]

But, what is actually uploading the main.yaml workflow artifacts to erlang.org?

It is uploaded from our internal build system. We cannot use the one that github actions builds as it needs to be signed in order to avoid problems with Windows thinking it is malware.

I think the best way to make the checksums is by implementing a new workflow that triggers on release updates that adds the correct md5/sha256 files. It cannot be part of the main.yaml flow as the installer is not guaranteed to be available when that flow runs.

[rgl]

But, what is actually uploading the main.yaml workflow artifacts to erlang.org?

It is uploaded from our internal build system. We cannot use the one that github actions builds as it needs to be signed in order to avoid problems with Windows thinking it is malware.

The internal build system is just signing the binaries that are built in the main github actions workflow? Or is it building a whole new set of binaries?

Why not sign them in/from the github actions workflow?

I think the best way to make the checksums is by implementing a new workflow that triggers on release updates that adds the correct md5/sha256 files. It cannot be part of the main.yaml flow as the installer is not guaranteed to be available when that flow runs.

I'm not sure that will work. From what I understood, the github actions release is created in the main github actions workflow, so that might trigger before your internal build system had a chance to sign and upload the binaries to erlang.org.

[garazdawi]

The internal build system is just signing the binaries that are built in the main github actions workflow? Or is it building a whole new set of binaries?

It is a different set of binaries. We have similar system internally to what GitHub actions does, but which also does testing for all platforms. We then want to release the same binary installer as we used in testing (we have had issues where the releases installer did not work, but we did not find it in test as the same one was not used in testing).

Why not sign them in/from the github actions workflow?

I’m actually not sure if we are allowed by the windows signing rules to do that, and besides we want to use the same binary for tests, so we cannot for that reason either.

I'm not sure that will work. From what I understood, the github actions release is created in the main github actions workflow, so that might trigger before your internal build system had a chance to sign and upload the binaries to erlang.org.

My hope was that there would be an update event triggered to the release when the new windows artefacts are added. If that is not the case, then I suppose the sync script will have to do it. Or a manually triggered workflow (that we trigger automatically, just as for erlang.org).