apache
[Task] Publish Prism binary (os + architecture combos) artifacts on Beam Release.
Eventually we should publish OS + Architecture specific binaries for Prism in order to make it easier to use with released versions, and avoid requiring users compile it locally with Go (if possible). Though, the "build locally" solution can always remain a fallback, in particular for dev use, or novel OS + Arch combinations we don't release.
Maintained Actions exist: https://github.com/marketplace/actions/go-release-binaries (would need to be INFRA approved, if not already) (More investigation into alternatives is recommended).
Otherwise, programmatic access exists for uploading to a release, and then downloading release assets: https://docs.github.com/en/rest/releases/assets?apiVersion=2022-11-28#upload-a-release-asset
https://docs.github.com/en/rest/releases/assets?apiVersion=2022-11-28#get-a-release-asset
There's no limit/quotas associate with github assets, outside of the 2GB per file: https://docs.github.com/en/repositories/releasing-projects-on-github/about-releases#storage-and-bandwidth-quotas
Then ultimately incorporating it into the release workflow for use with RCs and then the final releases.
Investigated goreleaser https://goreleaser.com/
And it would be perfect if Beam had a pure go project repo, as our needs aren't too complicated. But it can do complicated, and handle basically everything to do with a release, from change log handling, to release notes, uploading a release to GitHub, building and publishing docker images, signing, etc.
The part that fails to work for the current beam repo is that the Go SDK uses a sub version tag to work with Go Modules properly (that is it's released under the tags "sdks/v1.2.3", to be published properly. That requires the Pro version of go-releaser which requires a subscription fee.
That would enable even more sophisticated workflows, (like re-using the RC candidate binaries for the final tag release), but that would be overly optimizing. However, we would be better off leveraging existing Github mechanisms instead (eg. Create a release on the RC, but mark it as a pre-release).
We can revisit this if we move the Go SDK to it's own repo in some future or alternative version of beam.
Per that linked PR, we ended up just building an action for it ourselves.
Ultimately, we have the zipped files uploaded, along with signatures and hashes, so it fully matches what we upload to the Apache SVN repository.
The binary zips we're putting up in GitHub release artifacts will be at URLs with the following pattern. http://github.com/apache/beam/releases/download/RELEASE/apache_beam-RELEASE-prism-OS-ARCH.zip
Note: Beam versions have a v prefix, eg v2.56.0 which needs to match in the URL.
To actually finish #28187 we need to have Java (#31402) and Python (#31403) download the binaries from GitHub. For that just using whatever native library that can make the http request and handle the various redirects GitHub puts us through.