Standardize automatic NPM publishing

[lidel]

This is a placeholder issue for figuring out interplanetary conventions for automatic publishing to NPM. @hugomrdias lmk if you have any preference here, below is my summary of past conversations

Current state

• We have competing conventions and publishing flows across JS libs (IPFS/libp2p/ipld/multiformates):
  • JS IPFS libs usually require maintainer to do manual publishing via aegir release
  • Various IPLD repos use mikeal/merge-release + Github Action to publish to NPM on every merge to the main branch (major, minor or patch sem. version is picked based on commit messages)

Unifying publishing across JS repos

Details TBD, but during triage discussion we hinted at upstreaming some convention, perhaps to aegir/CI template, just like we did for tests and other checks in https://github.com/ipfs/aegir/blob/master/md/github-actions.md

Hard requirement: adopted convention should work fine in repos without aegir.

Prior Art

• mikeal/merge-release avoid commits on purpose
  • merge-release had some problems recently dealing with >1 commits in a merge and figuring out the right semver bump (a major ended up as a patch and required a scramble to fix – annoying, but fixable)
• release-please makes a PR so the maintainer is the one making the commits.
  • This hybrid approach gives maintainer a bit of control (to batch changes together or to release every commit), and makes it possible to eyeball what the next release will be, before it's released.
  • Seems to be blocked on support for npm workspaces/monorepos like js-ipfs one
  • Release-please also has an extra tool that allows 2FA to be on and avoid automation tokens. But you need to configure it and deploy it to Google engine thing and that will do the 2FA for you.
  • @hugomrdias did a PoC in hugomrdias/playwright-test where a PR is created from any outstanding changes and publish to NPM occurs when the aggregator PR is merged.
• semantic-release is what lerna uses behind the scenes, so it's being used by js-ipfs – we can see what the generated release notes look like, etc.
  • note from @achingbrain regarding js-ipfs monorepo:

    Whatever solution we settle on, it should support monorepos as well as single-module repos, that way we can use npm 7 workspaces for dep hoisting and running scripts in packages, and this tool for releases and we can drop lerna. Lerna works and works well, but npm 7 is faster and the long term maintenance of the project has looked a bit wobbly for a while.

[hugomrdias]

We also have a mono-repo setup here https://github.com/ipfs-shipyard/nft.storage with different types of packages that need different kinds of post release tasks using release-please

[rvagg]

I've seen https://semantic-release.gitbook.io/semantic-release/ a lot for this but have yet to try it out. I've been meaning to try it on some personal projects and will report back if I get around to that.

I'm about to get some experience with release-please as it was enthusiastically suggested for node-gyp where we have merge & release bottlenecks (the main bottleneck being me) but I've imposed a requirement that we not do automated releases yet because it's such a critical project that I don't want it to be too easy for a rogue commit to go straight out to npm without enough eyes. But we may iterate toward auto-releasing eventually with some confidence. First PR generated by it for release is: https://github.com/nodejs/node-gyp/pull/2401

[rvagg]

I just got semantic-release configured and working as I want it for ipld-garbage.

package.json config: https://github.com/rvagg/js-ipld-garbage/blob/828a551abe8a83a254b7abda2eb3775b1d42ec3b/package.json#L59-L75

GitHub Actions roll-up test & maybe release: https://github.com/rvagg/js-ipld-garbage/blob/master/.github/workflows/test-and-release.yml (release if it's running against master).

In this case it uses the new ipld packaging format which publishes from a built dist/ directory rather than ., and that's supported by the semantic-release npm plugin. So this gives me the same functionality as mikeal/merge-release and a bit more (I get changelogs and GitHub Releases published now!), plus getting access to an ecosystem of plugins.

Happy with this so far, I might roll it out further.

[rvagg]

some additional tweaks to get it to deal better with conventional-commits: https://github.com/rvagg/js-ipld-garbage/blob/master/package.json#L59-L96

[lidel]

Next step is to create a template repo with setup that covers our requirements (support for mono repo etc). Hopefully this is something a JS tribute could land within a week shift.

[achingbrain]

The template is coming together here: https://github.com/protocol/.github/issues/273

[TinyTb]

Done via https://github.com/protocol/.github/issues/273