stacks-core
stacks-core copied to clipboard
stacks-network
Update github action CI workflow
Updates the CI workflow
Applicable issues
- related PR for clippy (@igorsyl): https://github.com/stacks-network/stacks-blockchain/pull/3186
- fixes https://github.com/stacks-network/stacks-blockchain/issues/3148
Additional info
Lots of proposed changes here so I'll do my best to articulate them:
- migrates the existing ci.yml workflow into a more modular approach with semi-reusable workflow files
- updates all Debian builds to use bullseye (latest debian)
- adds arm64 binaries/docker image
- update all Dockerfiles to use ramdisk for building
- create checksum of binary artifacts for release
- build all release images from binaries (PR's and branch builds still build from source)
- create checksum of binary artifacts for release
- adds clippy and crate advisory actions
The workflow behaves similarly to how the existing workflow runs - manual execution with a tag will create a new release. on PR's, a simplified workflow is run.
The ./build-scripts Dockerfiles are all updated to be resuable but also renamed to reflect how the arches appear in the build process, i.e. linux-glibc-arm64.
Clippy has been added, and right now it's set to run on PR's to develop - as of now this action is quite messy. Opinions here are welcome, it may be desired to remove entirely, or perhaps run it on a cron so the data is available. Using defaults, it's very pedantic but it can be adjusted to ignore trivial lint complaints.
A crate advisory action is also added (I believe it's something that should remain in some form to ensure we're keeping up with dependency updates) - currently this action is to be scheduled nightly at 0330 UTC, and will create a new issue if any crate has a listed advisory.
Last, the build process is changed slightly:
- sha1sum of all archives is stored in the file
CHECKSUMS.txtwhich is included in the github release - All release images are built using the binary archives for the specified architecture
./github/actions/dockerhub/**is created to build the new images for both Alpine and Debian
Caveats
- the linux-musl-arm64 build is using an external musl-cross build image (for now), no other way to natively build for this arch. has been found (https://musl.cc/)
bitcoin-tests.ymlis set to run only when PR's are opened to the master branch (this is debatable, but in testing i felt it was running too often)- source build will trigger on a push to the default branch automatically
- naming of the tags is slightly adjusted from something like
x.y.z-stretch->x.y.z-debian
Examples of builds
- Example images built during testing: https://hub.docker.com/repository/docker/wileyj/stacks-blockchain/tags?page=1&ordering=last_updated
- Issues created from crate advisories: https://github.com/wileyj/stacks-blockchain/issues
- Test build of tag
2.05.0.2.3: https://github.com/wileyj/stacks-blockchain/actions/runs/2673483049 - Merge to master: https://github.com/wileyj/stacks-blockchain/actions/runs/2673477039
- Example clippy action: https://github.com/wileyj/stacks-blockchain/actions/runs/2673230254
One more decision I made in this PR (that I did not use for testing) - the jobs all use the current model like this:
uses: stacks-network/stacks-blockchain/.github/workflows/stacks-blockchain-tests.yml@master
so i left it as-is. however, this does make it hard to make changes to the ci workflow since a push to master is required to get changes into github actions.
during testing, i was using the local path to the action, ex:
uses: ./.github/workflows/stacks-blockchain-tests.yml
Both methods have their pros/cons - I felt it was important to note that here in this PR.
- How to tag a release (I think this is already kind of mentioned in the release checklist, but it could have it's own subsubsection in the README), and what happens automatically when the release is tagged (docker images pushed? binaries included in the git release? changelog?)
- How to trigger a "feature" build: if I have a PR, and I want to publish a docker image with a specific tag (not a version number) that doesn't create a new release, how can I do that? I think before these changes, that was done "automatically" on every
feat/...PR, but with these changes, I think it requires a manual trigger, right?
sorry, i missed this comment when i was addressing other changes. I'll check and reply later, optionally with a commit to address the questions
These changes look good to me -- I'll defer to Charlie and Jude for final approvals.
My only request is that you update the README.md (or add a RELEASING.md) with two instructions:
- How to tag a release (I think this is already kind of mentioned in the release checklist, but it could have it's own subsubsection in the README), and what happens automatically when the release is tagged (docker images pushed? binaries included in the git release? changelog?)
- How to trigger a "feature" build: if I have a PR, and I want to publish a docker image with a specific tag (not a version number) that doesn't create a new release, how can I do that? I think before these changes, that was done "automatically" on every
feat/...PR, but with these changes, I think it requires a manual trigger, right?
- Releases are only tagged if a tag is manually provided when the action is triggered -> this PR doesn't deviate from the current process. Updating the README.md might be better suited in https://github.com/stacks-network/stacks-blockchain/issues/3196 and https://github.com/stacks-network/stacks-blockchain/pull/3247
- Pushing a new feature branch -> nothing is triggered automatically. PR's are required (same process as current), or the ci workflow can be triggered manually on a specific branch to build assets.
PR a branch to develop:
- rust format is run (this happens on every run)
- docker image built from source on a debian distribution is built and pushed with the branch name and PR number as tags
blockstack/stacks-blockchain:feat-112-fix-somethingblockstack/stacks-blockchain:pr-112
Merging a PR to develop:
- rust format is run (this happens on every run)
- docker image built from source on a debian distribution is built and pushed with the branch name and PR number as tags
blockstack/stacks-blockchain:developblockstack/stacks-blockchain:pr-113
Merging a PR develop to master:
- rust format is run (this happens on every run)
- docker image built from source on a debian distribution is built and pushed with the branch name as a tag
blockstack/stacks-blockchain:master
Manually triggering workflow without tag (any branch):
- rust format is run (this happens on every run)
- no binaries are built
- no github release
- no integration tests
- docker image built from source on a debian distribution is built and pushed with the branch name as a tag
blockstack/stacks-blockchain:<branch name>
Manually triggering workflow with tag (non-default branch):
- rust format is run (this happens on every run)
- no binaries are built
- no github release
- no integration tests
- docker image built from source on a debian distribution is built and pushed with the branch name
blockstack/stacks-blockchain:<branch name>
Manually triggering workflow with tag on default branch (i.e. tag of 2.05.0.3.0):
- rust format is run (this happens on every run)
- integration tests
- leaked credential test
- binaries built for specified architectures
- archives and checksum added to github release
- github release (with artifacts/checksum) is created using the manual input tag
- docker image built from source on a debian distribution is built and pushed with the provided nput tag and
latestblockstack/stacks-blockchain:2.05.0.3.0-debianblockstack/stacks-blockchain:latest-debianblockstack/stacks-blockchain:2.05.0.3.0blockstack/stacks-blockchain:latest
TL;DR
Creating a release doesn't deviate from the current process. Because the builds for a release will now use a binary vs building from source for each arch, the debian-based source build docker image is not created when a new release is tagged. a release tag will produce 2 versions of the docker image (along with all binary archives):
- Alpine image tagged with the
x.x.x.x.xandlatest - Debian image tagged with
x.x.x.x.x-debianandlatest-debianThe debian-based docker image is created when feature branches etc are PR'ed (or manually triggered via the CI workflow, choosing the appropriate branch), resulting in a docker image being pushed with a tag likefeat-fix-1234
clippy workflow is being added, but it's currently disabled until we can tame it a bit so it's not overly verbose.
audit workflow is being added, but it's currently disabled. this workflow is not triggered unless some conditions are met (Cargo.toml/Cargo.lock are changed); it is also configured to run on a schedule and create issues for crate advisories. I've disabled it for now so we can evaluate if it's worth keeping (i feel that it is, but in the short-term it may add extra noise to the open issues).
Can you add that info either to the README.md or a new RELEASING.md file? That way, we'll have a place to look if we want to figure it out down the line.
Can you add that info either to the
README.mdor a newRELEASING.mdfile? That way, we'll have a place to look if we want to figure it out down the line.
Can do - will add a new file and bump the issue dealing with the readme updates
Can you add that info either to the
README.mdor a newRELEASING.mdfile? That way, we'll have a place to look if we want to figure it out down the line.
https://github.com/wileyj/stacks-blockchain/commit/97ae5638635cae3c8e0d61c866871954a32fa9c7
@CharlieC3 i think there's one more change i have to make here that i missed in my tests. when a release candidate is created, it should create all binaries/release/images and currently it does not - only realized this as i'm working on the current 2.05.0.3.0 release and watching the CI process
Merging a PR to develop: rust format is run (this happens on every run) docker image built from source on a debian distribution is built and pushed with the branch name and PR number as tags blockstack/stacks-blockchain:develop blockstack/stacks-blockchain:pr-113
Why would merging a PR to develop (effectively making a commit to develop and closing a PR) render a new PR Docker tag? I would think this operation should only create a tag where the new commit/merge was made (e.g. blockstack/stacks-blockchain:develop)
Manually triggering workflow with tag (non-default branch): rust format is run (this happens on every run) no binaries are built no github release no integration tests docker image built from source on a debian distribution is built and pushed with the branch name blockstack/stacks-blockchain:
Sometimes we create release-candidate Github tags off the develop branch. With this change, I'm not sure we'll be able to do that anymore. Is that intended?
Merging a PR to develop: rust format is run (this happens on every run) docker image built from source on a debian distribution is built and pushed with the branch name and PR number as tags blockstack/stacks-blockchain:develop blockstack/stacks-blockchain:pr-113
Why would merging a PR to develop (effectively making a commit to
developand closing a PR) render a new PR Docker tag? I would think this operation should only create a tag where the new commit/merge was made (e.g.blockstack/stacks-blockchain:develop)Manually triggering workflow with tag (non-default branch): rust format is run (this happens on every run) no binaries are built no github release no integration tests docker image built from source on a debian distribution is built and pushed with the branch name blockstack/stacks-blockchain:
Sometimes we create release-candidate Github tags off the
developbranch. With this change, I'm not sure we'll be able to do that anymore. Is that intended?
there are a few typos in the comment i made updated here: https://github.com/wileyj/stacks-blockchain/blob/feat/update-ci/RELEASING.md
however, your second point is correct and only something i noticed myself the other day (https://github.com/stacks-network/stacks-blockchain/pull/3199#issuecomment-1219839987 ) . it needs to be fixed so i'm prepping a change so that still works as exected (currently, trying to cut a release from a non-default branch wouldn't actually create the release, just the debian image).
Once i get those changes committed, it will also update the RELEASING.md file.
I'll ping you again when I think it's ready, it was a use case i wasn't thinking about when i sent this PR.
Merging a PR to develop: rust format is run (this happens on every run) docker image built from source on a debian distribution is built and pushed with the branch name and PR number as tags blockstack/stacks-blockchain:develop blockstack/stacks-blockchain:pr-113
Why would merging a PR to develop (effectively making a commit to
developand closing a PR) render a new PR Docker tag? I would think this operation should only create a tag where the new commit/merge was made (e.g.blockstack/stacks-blockchain:develop)Manually triggering workflow with tag (non-default branch): rust format is run (this happens on every run) no binaries are built no github release no integration tests docker image built from source on a debian distribution is built and pushed with the branch name blockstack/stacks-blockchain:
Sometimes we create release-candidate Github tags off the
developbranch. With this change, I'm not sure we'll be able to do that anymore. Is that intended?
https://github.com/stacks-network/stacks-blockchain/pull/3199/commits/663d8a71360a04b5f28c6bdec708a000f4b7a700
I've updated the RELEASING file to reflect the (tested) changes and what happens: https://github.com/wileyj/stacks-blockchain/blob/feat/update-ci/RELEASING.md
I think i have the desired behaviour, if i missed something let me know and i'll re-adjust.
Note: let's wait until after 2.05.0.3.0 before merging this, since it's potentially disruptive to the release process.
TODO: see if we can retry the codecov upload, or time out if it's not going to work
Since we are seeing these codecov errors very frequently, can we just change this setting to false in the various places it is used? At least until we come up with a better solution for code coverage.
fail_ci_if_error: true
Now that 2.1 is official, i'll be updating my fork for this. One thing i'll be adding to the build changes is the docs PR action failing (posisbly in a separate PR since i think that will be a small fix to get it working again after the docs repo refresh - appears a path the workflow was expecting no longer exists)
@wileyj I'm eager to use official multi-arch docker images in the API repo.
Should this PR target master since it's a release/CI related change and not a stacks-node code change?
@wileyj I'm eager to use official multi-arch docker images in the API repo.
Should this PR target master since it's a release/CI related change and not a stacks-node code change?
Not a bad idea. i've moved this PR to draft for now while I update downstream and get this in good shape again for review
These changes look good to me -- I'll defer to Charlie and Jude for final approvals.
My only request is that you update the README.md (or add a RELEASING.md) with two instructions:
- How to tag a release (I think this is already kind of mentioned in the release checklist, but it could have it's own subsubsection in the README), and what happens automatically when the release is tagged (docker images pushed? binaries included in the git release? changelog?)
- How to trigger a "feature" build: if I have a PR, and I want to publish a docker image with a specific tag (not a version number) that doesn't create a new release, how can I do that? I think before these changes, that was done "automatically" on every
feat/...PR, but with these changes, I think it requires a manual trigger, right?
moved this to ./docs/ci-release.md in line with recent changes to the repo.
Since we are seeing these codecov errors very frequently, can we just change this setting to false in the various places it is used? At least until we come up with a better solution for code coverage.
fail_ci_if_error: true
done, matching what's currently in the master branch