Update release workflow

[wileyj]

Note: relies on composite PR https://github.com/stacks-network/actions/pull/91 due to the changes in registry.

This change refactors the release workflow as follows:

• changes the binary build process from using docker to building on the VM directly
• docker images use the binary artifact archive, rather than assets from a published github release
• multiple approval gates are removed in favor of a single one to start the build
• docker images publish to ghcr (this will require some additional setup after merge)
• workflows and inputs have been simplified to reduce duplication and remove variables that aren't needed
• docker tags are simplified: stacks-blockchain is no longer tagged by default for ghcr
• workflow filenames have been renamed slightly to better reflect what they do. release-*.yml etc.
• building the ad-hoc docker image using the same artifact method, but is coped only for glibc x64/arm64
• all archives and images are attested, with the docker images also being signed with cosign (https://github.com/sigstore/cosign)
• dockerfiles used to create the images are re-organized based on target and purpose
• macos builds are building on macos runners, with the only cross-compiliation being for cpu/c stdlib, which only involves updating linkers and other build-time flags.
• removed the cleanup step when setting up docker. via composite recent runners don't benefit from this, and the storage is more than adequate now to build stacks-core.

Overall, the performance is much better than using docker - what was taking ~30 minutes (discounting timeouts and failures) is now reliably down to ~12 minutes per build (macos being the outlier, since it was tested on a 7GB machine with the release-lite profile).

a full test release workflow: https://github.com/wileyj/stacks-core/actions/runs/19481419227

and a specific run of just the release workflow to show the expected timing of the changes proposed here: https://github.com/wileyj/stacks-core/actions/runs/19475022815

and the ghcr images produced- signer: https://github.com/wileyj/stacks-core/pkgs/container/stacks-signer/versions?filters%5Bversion_type%5D=tagged stacks-core: https://github.com/wileyj/stacks-core/pkgs/container/stacks-core/versions?filters%5Bversion_type%5D=tagged

published test release: https://github.com/wileyj/stacks-core/releases/tag/4.0.0.0.6 published singer test release: https://github.com/wileyj/stacks-core/releases/tag/signer-4.0.0.0.6.0

finally - the builds are set to use the release profile, which willl require at least the macos-latest-large runner, with the linux runners currently set to use the default ubuntu-latest (this may be changed, but the timing on default runners was acceptable).

the binaries/images need more verification, but initial testing of produced artifacts looks good.

[codecov[bot]]

Codecov Report

:white_check_mark: All modified and coverable lines are covered by tests. :white_check_mark: Project coverage is 76.86%. Comparing base (39a0bbb) to head (573990b). :warning: Report is 50 commits behind head on develop.

Additional details and impacted files

@@             Coverage Diff              @@
##           develop    #6697       +/-   ##
============================================
+ Coverage    52.86%   76.86%   +24.00%     
============================================
  Files          578      579        +1     
  Lines       358379   360068     +1689     
============================================
+ Hits        189458   276772    +87314     
+ Misses      168921    83296    -85625     

see 445 files with indirect coverage changes

---

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 39a0bbb...573990b. Read the comment docs.

:rocket: New features to boost your workflow:

• :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• :package: JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[wileyj]

moving to ghcr - one thing i don't like is how all the layers (tagged or not) are visible on the packages page (see link above). i have a feeling it's a UI diff compared to something like dockerhub, but it may be confusing for users looking for what is the latest tagged release.

a few things to follow up on:

• schedule a nightly build from develop. i think this should be easier than expected with the move to using artifacts built natively. a simple conditional tag if it's run via cron should be all that's needed (along with hardcoding to build nightly from develop, compared to any desired branch when run manually)
• github release is created in draft mode, but i think we could also pre-populate most of the release message from the changelog(s) using a tool like https://github.com/sean0x42/markdown-extract

[wileyj]

there is also a question of if we should be re-running all the same tests (which should have been passing prior to release in order for the changes to merge), i.e. bitcoin tests. on the other hand, there could be a case where a hotfix release breaks an existing test, and it woudn't be caught until the release ci.

[wileyj]

update - one thing i notice that i'm not sure about (but shoudln't be a blocker to merge) is that the artifacts are being attested 2x for the same archive. this may be a result of the the archive name being identical for stacks-core and the stacks-signer release.

Edit: it is indeed because of the archive file naming being a duplicate for the release. so stacks-core & stacks-signer both having the same archive name like linux-glibc-x64.zip will produce 2 quite similar attestations, even though the context of that archive is different.

[fabrizio-stacks]

update - one thing i notice that i'm not sure about (but shoudln't be a blocker to merge) is that the artifacts are being attested 2x for the same archive. this may be a result of the the archive name being identical for stacks-core and the stacks-signer release.

Edit: it is indeed because of the archive file naming being a duplicate for the release. so stacks-core & stacks-signer both having the same archive name like linux-glibc-x64.zip will produce 2 quite similar attestations, even though the context of that archive is different.

if you have a look to the sbtc repo I solved this problem!

[federico-stacks]

there is also a question of if we should be re-running all the same tests (which should have been passing prior to release in order for the changes to merge), i.e. bitcoin tests. on the other hand, there could be a case where a hotfix release breaks an existing test, and it woudn't be caught until the release ci.

just to clear my understanding: does it means we don't run the "ci" tests for the hotfix release?

[wileyj]

there is also a question of if we should be re-running all the same tests (which should have been passing prior to release in order for the changes to merge), i.e. bitcoin tests. on the other hand, there could be a case where a hotfix release breaks an existing test, and it woudn't be caught until the release ci.

just to clear my understanding: does it means we don't run the "ci" tests for the hotfix release?

no - just a note that we currently re-run all tests that are run as part of the PR process when doing a release, and perhaps there's an option to not do that in the future (with the expectation that tests would have already run. there are some caveats today, but i think it's possible).

for a hotfix release, we'd still need to run all the normal required steps. but, i think there may be a case to be made where we can ensure all tests are passing before a release workflow is triggered - not addressed in this PR, but something to think about for a future change. for hotfixes, the tests should likewise already be run by the time the workflow is triggered. it's a good question though, i think it's worth the effort to determine when and how we run tests.