Automated release workflow

[trask]

Hey all!

We've spent a lot of time lately improving our release workflow in the OTel Java repos, and I'd like to see if our toil and pain can be of any benefit to other repos (starting with python of course!).

I also think there's some benefit of shared cross-SIG knowledge going forward if we follow somewhat similar workflow practices.

I've tried to document things here: https://github.com/trask/repository-template/blob/main/template-docs/release-automation.md

Some highlights in this PR:

• Uses -dev version suffix on main (@lzchen mentioned you were thinking about doing this anyways)
• The release workflow autogenerates the release notes from the change log and publishes the github release
• Uses the bot workaround discussed in https://github.com/open-telemetry/community/issues/809#issuecomment-1015121722 (maybe we should consider creating a shared otel bot that all repos could use?)
• The branch names are based on the minor release (e.g. release/v1.12.x-0.31bx), and patch releases are made out of those same branches. Tags are still applied to each individual release.

New complications in this repo that we hadn't faced in the Java repos:

• Separate versions for stable/unstable components (this wasn't a big deal, just a bit more workflow code to handle/increment them both).
• I tried adding the Skip Changelog label to the generated pull requests that target main, but that requires giving the bot triage rights to the repo, so instead suppressed the changelog workflow when github.actor is the bot.

I'd love to stop by your SIG meeting, but it's always at the same time as the Java SIG 😢. Maybe next week I can sneak away and jump to your meeting for a bit.

Please throw your questions this way though, I'd love to better understand where we can align more, and where things will naturally diverge based on SIG needs/preferences.

[srikanthccv]

This is nice

[trask]

I should have mentioned, I tested all of these workflows on my python repo fork, though there may still be a hiccup or two on the first try here. I'd be happy to coordinate and be online with you during the first release using these new workflows if you are interested.

[srikanthccv]

Sorry I haven't gotten a time to review this in detail but there are many things I like about this. Team has been focused on getting the rc for metrics work for the past few weeks. So it might take sometime before we review and get it merged.

[trask]

@lzchen thanks for all the comments! I have made several updates and I think replied to all of your comments, happy to make further updates

[srikanthccv]

@trask will it be possible for you to attend tomorrow python SIG meeting and present it to all?

[trask]

@trask will it be possible for you to attend tomorrow python SIG meeting and present it to all?

yes, I'll plan on it, would it work to join at 9:30am Pacific Time?

[srikanthccv]

That should work

[lzchen]

@trask

Will be approving after the rc case has been confirmed to work.

[trask]

I updated the workflow to work with "pre-release" versions, e.g. "1.13.0rc", and tested making a "pre-release" in https://github.com/trask/repository-template/releases/tag/v1.1.0rc.

I'll re-test the full workflow in my opentelemetry-python fork one last time once everything is approved, but first could use some help resolving the build error below:

ERROR: Cannot install opentelemetry-instrumentation-requests==0.31b0, opentelemetry-sdk==1.13.0.dev0 and opentelemetry-semantic-conventions 0.32b0.dev0 (from /home/runner/work/opentelemetry-python/opentelemetry-python/opentelemetry-semantic-conventions) because these package versions have conflicting dependencies.

The conflict is caused by: The user requested opentelemetry-semantic-conventions 0.32b0.dev0 (from /home/runner/work/opentelemetry-python/opentelemetry-python/opentelemetry-semantic-conventions) opentelemetry-sdk 1.13.0.dev0 depends on opentelemetry-semantic-conventions==0.32b0-dev opentelemetry-instrumentation-requests 0.31b0 depends on opentelemetry-semantic-conventions==0.31b0

To fix this you could try to:

1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

[srikanthccv]

Our current workflow for patch/minor/major release is pretty simple. There are few additional steps introduced in this which also eliminates few manual steps we do today. Overall LGTM, I want to test on this repo or fork repo once.

[trask]

oops! didn't mean to delete that branch 😅

I made some nice simplifications to the change log management based on conversations above.

I'm re-testing all the workflows on my fork right now and will post back when it's ready for another review

[trask]

I tested the release workflow in my fork for the next release, which I understand will be 1.12.0rc2/0.32b0 (it could also be 0.31b1, but it makes the release workflow cleaner if you reserve b1 for patch releases).

I will test the patch release workflow tomorrow (e.g. I'll simulate 1.12.0 and then a 1.12.1 on the 1.12.x release branch).

btw, check out this section I added to RELEASING.md to clarify the situation with prereleases:

Notes about "pre-releases"

• Pre-release versions (e.g. 1.12.0rc1-0.31b0) are supported, and will cause a "short-term" release branch to be created based on the full version name (e.g. release/v1.12.0rc1-0.31b0 instead of a "long-term" release branch name like release/v1.9.x-0.31bx).
• Patch releases are not supported on short-term release branches.
• The version in main in this case will be bumped to the release version (e.g. 1.12.0-dev/0.32b0-dev).
• To make a second pre-release version, manually update the version in main (e.g. update it from 1.12.0-dev to 1.12.0rc2-dev) before running the prepare release branch workflow for that release.
• Note that pre-releases are not needed for unstable artifacts.

[trask]

ok, I've re-tested all the workflows on my otel-python fork, I think this is ready for a final review

[lzchen]

@trask I'm testing this in my local repository.

For the two PRs that are generated from prepare-release-branch workflow, for the second pr that updates the version in main from 1.12.0rc2-dev to 1.12.0-dev, is this expected? I'm not too familiar with dev version conventions but should the version be the NEXT version? i.e. 1.12.1-dev or 1.13-dev? Something like that?

[lzchen]

@trask Just to verify, I performed the prepare-release-branch workflow one more time after the previous run above, and the first PR that merges into the release branch changes the version now from 1.12.0-dev to 1.12.0. I would think the new version would be 1.13.0 correct?

[trask]

For the two PRs that are generated from prepare-release-branch workflow, for the second pr that updates the version in main from 1.12.0rc2-dev to 1.12.0-dev, is this expected? I'm not too familiar with dev version conventions but should the version be the NEXT version? i.e. 1.12.1-dev or 1.13-dev? Something like that?

In Java land, we use -SNAPSHOT, which has special meaning in maven (the primary Java build tool), and whatever version we are working towards, e.g. 1.12.0, we append -SNAPSHOT to the version and our local build will get published as 1.12.0-SNAPSHOT.

If we are working towards 1.12.0-rc.2, then our local build would be 1.12.0-rc.2-SNAPSHOT, and when we are ready to release then we remove the -SNAPSHOT, and release 1.12.0-rc.2.

After releasing 1.12.0-rc.2, in Java land, we often bump to 1.12.0-SNAPSHOT, under the assumption that the next release will be the final 1.12.0 release (which comes after 1.12.0-rc.2 prerelease in semver ordering).

I guessed that -dev in python is similar to -SNAPSHOT, but I could definitely be off base.

Also I notice that your prerelease versions don't follow semver (e.g. 1.12.0rc1 instead of 1.12.0-rc1), not sure if that's a python thing, or if you'd like to change that.

If you can describe the versioning strategy that you would like to follow, I can update the workflows accordingly.

And if you don't want to mess with -dev that's not a problem, I can remove that. I assume in that case you'd want the version in main to be the version you are working towards? (without and -dev suffix)

[trask]

And if you don't want to mess with -dev that's not a problem, I can remove that. I assume in that case you'd want the version in main to be the ~version you are working towards~ prior version? (without and -dev suffix)

I looked at the opentelemetry-js and opentelemetry-go repos, and neither of them use -dev, so I'm thinking that's just a special Java thing (-SNAPSHOT).

In those two repos, they leave the version alone in the main branch until it's time to release and then they bump it. (so generally the version in main reflects the most recently released version).

Which looks like exactly what you are doing in this repo currently.

So I'm thinking it makes sense to remove the -dev stuff at least for now.

[trask]

I kept the -dev suffix for now, but improved the prerelease ("release candidate") handling:

• renamed "prerelease" in the workflow to "unstable", I think this is a clearer distinction between "stable" components vs "unstable" components instead of "stable" vs "prerelease" since you can have prereleases (i.e. RCs) of stable components
• added an optional input parameter when running the "prepare release branch" workflow where you can specify a prerelease version (for the stable components), this improves a couple of things:
  • no longer need to manually update the version on main to 1.9.0rc1-dev before running the workflow when you want to make a prerelease
  • no longer have the weird "prerelease of a prerelease" version in main 1.9.0-rc1-dev
  • so now the version in main remains 1.9.0-dev, which is a (semver) prerelease of 1.9.0, so makes more sense I think

[trask]

I will do another full test of the workflows on my fork, but will wait a bit to see if you would like any other behavioral changes first

[trask]

I switched the order to publish to PyPI before publishing the GitHub release.

This is what we've done in Java, since publishing to Maven is more likely to fail, but I can switch the ordering back if you'd like.

[lzchen]

@trask

Also I notice that your prerelease versions don't follow semver (e.g. 1.12.0rc1 instead of 1.12.0-rc1), not sure if that's a python thing, or if you'd like to change that.

It is a python specific thing: https://peps.python.org/pep-0440/#pre-releases

[lzchen]

@trask

So I'm thinking it makes sense to remove the -dev stuff at least for now.

I think adding dev is a decision made by the Python SIG because we won't have a way to conveniently test local changes. We would want to add that regardless of this workflow pr. However to not block this, you can assume that dev is not part of our workflow and we will fix the workflow once it is added.

[trask]

It is a python specific thing: https://peps.python.org/pep-0440/#pre-releases

nice, thx for sharing the link 👍

I think adding dev is a decision made by the Python SIG because we won't have a way to conveniently test local changes

I see in the same PEP 440 they also define the versioning for Developmental releases, I will update the workflow to conform to that convention 👍

[trask]

oh, I also created a new bot account @opentelemetrybot for everyone to use, and updated the workflow to use that

I will share the Personal Access Token with you, and I'll register the bot account at https://github.com/open-telemetry/community/blob/main/assets.md

[trask]

I added documentation in RELEASING.md to better explain the version numbering:

Notes about version numbering for stable components

• The version number for stable components in the main branch is always X.Y.0.dev, where X.Y.0 represents the next minor release.
• When the release branch is created, you can opt to make a "pre-release", e.g. X.Y.0rc2.
• If you ARE NOT making a "pre-release":
  • A "long-term" release branch will be created, e.g. release/v1.9.x-0.21bx (notice the wildcard x's). Later on, after the initial release, you can backport PRs to a "long-term" release branch and make patch releases from it.
  • The version number for stable components in the release branch will be bumped to remove the .dev, e.g. X.Y.0.
  • The version number for stable components in the main branch will be bumped to the next version, e.g. X.{Y+1}.0.dev.
• If you ARE making a "pre-release":
  • A "short-term" release branch will be created, e.g. release/v1.9.0rc2-0.21b0 (notice the precise version with no wildcard x's). "Short-term" release branches do not support backports or patch releases after the initial release.
• The version number for stable components in the main branch will not be bumped, e.g. it will remain X.Y.0.dev since the next minor release will still be X.Y.0.

Notes about version numbering for unstable components

• The version number for unstable components in the main branch is always 0.Yb0.dev, where 0.Yb0 represents the next minor release.
  • Question: Is "b" (beta) redundant on "0." releases, or is this a python thing? I'm wondering if we can change it to 0.Y.0 to match up with the practice in js and go repos.
• Unstable components do not need "pre-releases", and so whether or not you are making a "pre-release" of stable components:
  • The version number for unstable components in the release branch will be bumped to remove the .dev, e.g. 0.Yb0.
  • The version number for unstable components in the main branch will be bumped to the next version, e.g. 0.{Y+1}b0.dev.

[srikanthccv]

@trask since we had released rc2 I think we can get this reviewed and merged, please resolve the conflicts.

[trask]

merge conflicts resolved, ptal 👍

[srikanthccv]

There have been many changes since the last time. Does it make sense to revive this now? I think we don't any other reason to put this on hold.

[ocelotl]

There have been many changes since the last time. Does it make sense to revive this now? I think we don't any other reason to put this on hold.

So, you see value in reviving this, @srikanthccv? (I do)

[trask]

Rebased and re-tested the release process (both minor and patch) in my fork.

I realized that the backport automation (for patches) will fail currently most of the time due to change log conflicts. I think I can make the backport automation smarter and handle that, but I don't think it's necessarily required for landing this PR, as backports for patch releases can still be done manually.