[rush] Make Node and Rush installation hermetic

[lhorie]

Is this a feature or a bug?

• [x] Feature
• [ ] Bug

Please describe the actual behavior.

Currently, Rush can download and manage the version of the package manager, but not of Node and Rush itself. This means different team members could run into machine-specific issues if they have different versions of Node (and to a lesser extent Rush) installed.

This is something Bazel (Google's monorepo tooling) is able to support relatively well.

[alexeagle]

Maybe Bazel and Rush should be best friends :)

I'm the owner of rules_nodejs (canonical JS/Node support in Bazel) and happy to discuss!

[octogonz]

Currently, Rush can download and manage the version of the package manager, but not of Node and Rush itself. This means different team members could run into machine-specific issues if they have different versions of Node (and to a lesser extent Rush) installed.

@lhorie That's not entirely true. The Rush version you get is determined by the rushVersion setting in rush.json for each repo.

For example, suppose a long time ago you ran npm install -g @microsoft/rush and you got Rush version 5.9.1. This globally installed binary is just a minimal launcher. If you now run rush -h inside a repo folder with "rushVersion": "5.14.0", it will transparently download and invoke the engine for Rush 5.14.0. You won't see command-line help for the 5.9.1 release; instead you'll see all the new parameters for 5.14.0. In fact the launcher binary for 5.14.0 and 5.9.1 has hardly any changes, since its job is really simple. Even a very old launcher is likely to work just fine in a repo with the latest rushVersion, and vice versa. Thus, the Rush versioning is 100% deterministic.

It's true that we don't do this with NodeJS. Instead, repositories specify a nodeSupportedVersionRange in rush.json that restricts developers to a range of versions. It's common practice to restrict to a NodeJS major range that is designated as Long Term Support, which in our experience is relatively unlikely to have breaking changes. It would be ideal for Rush's NodeJS installation to be perfectly deterministic, however insisting on a specific version might cause friction for typical contributors. One reason is that people commonly choose many different ways of installing NodeJS (nvm, nvm-windows, nodist, etc). In practice, we very rarely see regressions due to minor/patch increments within an LTS range of NodeJS, so I'm not sure if it's worth improving or not.

[octogonz]

Bazel seems pretty interesting. I need to read more about it.

One potential downside is that it seems a bit complicated to install. You need an installer with prebuilt binaries, as well as a number of prerequisites. The "standard" for NodeJS is to require only NodeJS and VS Code. Minimizing prerequisites makes life easier for people who contribute to many different GitHub projects.

[lhorie]

The Rush version you get is determined by the rushVersion setting in rush.json for each repo.

Ah, interesting. Thanks for the correction.

It's been a while since I ran into issues so forgive me if my details are fuzzy, but IIRC, the issue I ran into was that some CI machines we had were running on Node 0.10.x (yes, I know.... don't ask) and that was causing problems w/ bootstrapping Rush.

Bazel itself actually doesn't manage its own version. There's another package that does that (called bazelisk). For a tool that we're developing at Uber, we wrap around that. The tool is here if you're curious, but basically it can manage its own version (like Rush does, plus that of Bazel, Node and Yarn). The corollary is that the bootstrap process for this tool only requires bash (as opposed to requiring a sufficiently recent Node to be installed to run install-run-rush).

We use shell scripts because we don't currently need Windows support, but maybe this code will be useful to you as a resource if you do want to pursue per-repo deterministic Node versions, or include Node installation itself in the boostrapping process.

FWIW, I do see issues going from different versions of node, even patch bumps, due primarily to GYP headers and network flakiness in CI systems, but I agree that this may not be the most common use case.

[alexeagle]

Bazel is mirrored to npm at @bazel/bazel and you can skip their install prerequisites (other than bash/msys on Windows right now, but we're working to remove that) So the way we use this in the node ecosystem is generally idiomatic and users don't need to know that @bazel/bazel is in the devDependencies

[alexeagle]

Update: Bazel (via https://github.com/aspect-build/rules_js) now uses pnpm internally just like Rush. Should make an easier migration path for anyone who wants more hermeticity.