beam icon indicating copy to clipboard operation
beam copied to clipboard

[#29697] Add prism artifact building workflow.

Open lostluck opened this issue 1 year ago • 2 comments

Adds a build & stage to RC creation to release signed hashed and zipped prism binaries to a release.

Currently requires a release with the RC tag to exist, actually straightening out the logic is best handled in a separate PR, as this change is sufficiently complex as is without including the clobber protection logic if targetting an already published release.

Changes the workflow dispatch to us a JSON object instead of individual string fields. The dialog now looks like this:

image

Not an ideal experience, but we hit the 10 input limit otherwise.

The action syncs down the Dev Apache SVN repo, builds prism binaries for ARM64, AMD64 for Linux, Darwin (osx), and Windows, zipping up the binary, creating signature and sha512 hash files.

These are uploaded to a Github prerelease for the RC tag, like so: https://github.com/apache/beam/releases/tag/untagged-819e4037252d95b05550

Finally, the same artifacts are committed to the Dev Apache repository for final publishing.

Signature and hash are not included in the Zip so the archive doesn't need to be decompressed before being able to validate it.

TODO in a subsequent PR:

  • automatically create a draft prerelease release for the tag if one doesn't exist yet.

  • Update existing draft GH release for the release version with the new RC tag.

  • Fail early if a published release has already gone out for the version to avoid clobbering an existing non-RC GH release.

  • Updating the Release Manager guide.

Part of #29697


Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

  • [ ] Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
  • [ ] Update CHANGES.md with noteworthy changes.
  • [ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md

GitHub Actions Tests Status (on master branch)

Build python source distribution and wheels Python tests Java tests Go tests

See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.

lostluck avatar May 21 '24 23:05 lostluck

R: @kennknowles cc: @damccorm @jrmccluskey @abacn

lostluck avatar May 24 '24 19:05 lostluck

Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control

github-actions[bot] avatar May 24 '24 19:05 github-actions[bot]

thanks. The refactoring makes sense as there are more and more fields in the action which is inconvenient. Just wondering if there is a way to keep the descriptions of field concise. Currently the dropdown is very long. (It's fine to keep as is for now)

Abacn avatar May 27 '24 18:05 Abacn

thanks, had two optional comments.

Do we plan to deliver prism binaries for Beam 2.57.0 (release cut soon)

Yup. That's why I was working on this all last week, to figure out and solve problems before we ran into them in the release.

It's easier to get the SDK downloading step working with a proper release having the artifacts. That half is for 2.58.0 though, and not 2.57.0.

lostluck avatar May 27 '24 18:05 lostluck

thanks. The refactoring makes sense as there are more and more fields in the action which is inconvenient. Just wondering if there is a way to keep the descriptions of field concise. Currently the dropdown is very long. (It's fine to keep as is for now)

Unfortunately I tried a few things for this and nothing made it better. We're ultimately going to need to keep the release guide documentation (which is in a later PR), paired with this and link to the documentation here to reduce the visual noise.

TBH, we'd probably move the initiation for this action to a cooy/pastable command line to do the workflow dispatch. See https://cli.github.com/manual/gh_workflow_run

lostluck avatar May 27 '24 18:05 lostluck

Since Kenn's going to affected by it, hopefully he'll review it first!

lostluck avatar May 27 '24 18:05 lostluck

Merging in, but I'm still going to be working with @kennknowles to make sure this is iterated on for completeness.

lostluck avatar May 29 '24 03:05 lostluck