Automating the release process using Github workflows

[bcdurak]

Describe changes

I implemented two new Github workflows and a bunch of scripts to automate our release process.

(The description of the PR mentions most things already. I will update it again when I have time.)

1. Workflow: release-prepare

Triggers the following jobs when a PR is opened from a misc/prepare-release-X branch.

• Prepares changes
  • Validates the new version
  • Sends a message to Discord
  • Creates and formats the Alembic migration
  • Updates README, pyproject, VERSION and helm files
  • Updates the references in example/quickstart
  • Auto generates the RELEASE_NOTES and appends it to the file
  • Pushes the changes to the misc/prepare-release-X branch
• Builds test images
  • Builds a base image using docker/zenml-dev.Dockerfile
  • Builds a server image docker/zenml-server-dev.Dockerfile
  • Builds a quickstart image per major cloud provider using docker/zenml-quickstart-dev.Dockerfile
  • Pushes the image to the target locations (AWS Quickstart image has to be hosted on AWS)
• Sets up the release prep tenant
  • We have a dedicated staging Release Prep tenant which is configured to use the image built by the previous job
  • It deactivates it and redeploys it so it will use the newly built server image.
  • (The stacks are already set up in this tenant.)
• Run Quickstart pipelines
  • Connects to the Release Prep tenant
  • Installs ZenML and the required integrations based on the cloud provider
  • Runs the quickstart example

2. Workflow: release-finalize

Triggers when a PR is merged from a branch with the misc/prepare-release-X pattern.

• Fetch versions
  • Fetch the new and old version (used as inputs to the following steps)
• Create the release branch
  • Checkout to develop
  • Create the release branch
• Add docs warning header
  • Checkout to release/old_version
  • Add the warning headers to the docs pages
  • Open up a PR
• Add the new version to migration tests
  • Checkout to develop
  • Check whether it is necessary to add the new version to the migration tests using alembic history
  • Open up a PR if that's the case
• Order gitbook spaces
  • Use the gitbook API to order the gitbook spaces between the docs and the legacy docs
• Deprecate the docs of the latest release on gitbook
  • Checkout to docs/legacy-docs-page
  • Adjust the cards to include the newly deprecated latest release with the proper link
  • Open up a PR

Remaining TODOs

• Test and beautify the discord message
• Test validate new version errors and warnings
• Test new version might fail if the latest version is 0.42.0rc

Pre-requisites

Please ensure you have done the following:

• [X] I have read the CONTRIBUTING.md document.
• [X] If my change requires a change to docs, I have updated the documentation accordingly.
• [ ] I have added tests to cover my changes.
• [X] I have based my new branch on develop and the open PR is targeting develop. If your branch wasn't based on develop read Contribution guide on rebasing branch to develop.
• [ ] If my changes require changes to the dashboard, these changes are communicated/requested.

Types of changes

• [ ] Bug fix (non-breaking change which fixes an issue)
• [X] New feature (non-breaking change which adds functionality)
• [ ] Breaking change (fix or feature that would cause existing functionality to change)
• [ ] Other (add details above)