Automate Releases

[apeltzer]

Since we already insist on having people work on dev branches & master branches only incorporating stable code, we could also produce a way of making automated releases too:

https://github.com/semantic-release/semantic-release

This would only require:

• Setting the master branch protected for everyone, except for PRs coming from dev
• Forcing everyone to NEVER merge to master except for releases that have been tested on dev (which we anyways do)

Could then configure the method above to make a release whenever coming from master and something has been changed. If people then also use the CHANGELOG.md, it would automatically do proper and nice releases :-)

[ewels]

Yes! And as just discussed on Slack, we could take advantage of this to do a load of nice extra automation:

Zenodo APIs

Zenodo DOIs are great, and the GitHub integration to generate them when GitHub releases are made works really nicely. However, the DOI is generated after the release is made, meaning that this new DOI can't be in the repo code. It is possible to reserve Zenodo DOIs, however not with the automated GitHub integration - it can be done with the API though, see https://developers.zenodo.org/#representation (see prereserve_doi).

With automated builds, we can have a GitHub Action that is triggered when dev is merged into master. This builds the release notes from the changelog, reserves a Zenodo DOI using the API, updates the repo and commits this new DOI, then makes the release and completes the Zenodo registration. 🚀

Pipeline version numbers

Pipeline version numbers need to be updated twice for every release - first for the release itself, then to bump to the next dev version. Typically the first of these just involves removing dev from the existing version number.

If automating releases, we could automatically run nf-core bump-version, stripping the dev off the existing version number for the release, then making a new commit to the dev branch with the next version number up (.dev).

...anything else we can automate?

[maxulysse]

Sounds quite interesting, I'd love to see that too :-D. Count me in ;-)

[apeltzer]

@olgabot had a good idea here: https://github.com/nf-core/tools/issues/382

[sven1103]

x-ref https://github.com/nf-core/tools/issues/361, because the CFF would need to be updated with the correct DOI before the release as well.

[sven1103]

Started to model the release process for nf-core pipelines with BPMN, so we have some documentation of it :D (not complete yet, but feel free to hit me with the processes that are missing)

The idea is that the GitHub Actions will follow this release process specification and make the process more transparent.

[apeltzer]

Should we also have a comment (using the citation file maybe?) that shows up in the README issuing the person using the pipeline to please also cite the DOI?

[apeltzer]

I think this looks pretty amazing already (thanks @sven1103 for drawing this out too)! I believe we can parse the citation file format to also add a section (when necessary, it might be already present there) to the readme to cite the proper DOI using GitHub Actions ?

That way we have both: an updated Readme with easy-to-read text for users, the standardized citation file format file for parsing/automated lookups, without the need to update the readme manually in such cases.

[sven1103]

Current status of the process (will be modified regularly until finished):

[ewels]

And automate the bibtex file too..

[drpatelh]

For reference, just seen this GitHub Actions code here