feat: download the latest version instead of a pinned one

[aduh95]

This PR ensures that fresh install of Corepack would always use the latest available version for a given package manager.

We probably want to add a command to clear the cache for lastKnownGood version (although I think that's more or less what corepack prepare is doing), and/or have a expiracy date for the lastKnownGood version to make Corepack update the default version from time to time.

Fixes: https://github.com/nodejs/corepack/issues/93

[zkochan]

Sorry for not answering. I think it is OK to ship the latest pnpm. Thanks

[andersk]

I have some concerns about this change.

1. This negates the hash protection of #137.

2. This could break the reproducibility of builds or CI jobs that start from a specific Node.js version (unless specifically overridden).

3. Consider a user who gets semi-regular automatic Node.js updates from their distribution, which means they get semi-regular automatic corepack updates. Under the previous scheme, they would therefore get semi-regular automatic updates to the default versions of Yarn and pnpm. But now those versions will be fixed from the first time they’re run (until a manual update).

[aduh95]

1. This negates the hash protection of chore: add sha1 to default versions when available #137.
2. This could break the reproducibility of builds or CI jobs that start from a specific Node.js version (unless specifically overridden).

The workaround is to set a specific version in the project package.json and/or set COREPACK_DEFAULT_TO_LATEST to 0 in the CI env.

3. Consider a user who gets semi-regular automatic Node.js updates from their distribution, which means they get semi-regular automatic corepack updates. Under the previous scheme, they would therefore get semi-regular automatic updates to the default versions of Yarn and pnpm. But now those versions will be fixed from the first time they’re run (until a manual update).

Contributions are welcome to improve this workflow.

[andersk]

Of course there are workarounds. But good defaults matter, and I’m proposing that the default should remain the secure known-good version.

A user who discovers they need a later version than the known-good version is in a position to go read the corepack documentation and configure it appropriately. (Most likely what they really want is to set a version in package.json so that everyone working on that project gets the right version. But if they want to set it system-wide, sure, they can have that option.)

Meanwhile, a user who unexpectedly gets a later (or earlier!) version when they didn’t make a change—they just re-ran the same script on a different system—is in a much more confusing position. They could very well be trying to get dozens of things working at the same time; they might not be an expert on the particular part of the script that failed, and might not even realize in the moment that their Yarn or pnpm is coming from corepack. Could they (or someone else who wrote the script) have had the foresight to reconfigure corepack more deterministically? Of course, but a trap that could be avoided is still a trap; good defaults matter.

Users don’t expect that installing a deterministic version of A should give them a nondeterministic version of B. We shouldn’t force them to remember or disable an exception to this rule.

[ljharb]

That’s exactly what users in this ecosystem expect - determinism, for better or worse, is simply not an expectation (beyond a lockfile), and anything else is surprising and very inconvenient, and in this case, causes many more people to have a buggy or insecure version of their package manager CLI.

[aduh95]

But good defaults matter, and I’m proposing that the default should remain the secure known-good version.

That's the catch, the known-good version is more likely to be less secure than the latest one.

I agree with Jordan, I don't think most users care about that kind of determinism (otherwise, they would set a specific version either system-wise, or at the project level). Also, as I wrote in https://github.com/nodejs/corepack/pull/134?notification_referrer_id=NT_kwDOANpZjbMzODgzNzkyMzM1OjE0MzA5Nzcz#discussion_r917276616, the reason Corepack is integrated in the Node.js project is that it make decouple what version of npm comes bundled with the Node.js installation, users can get a secure version of their package manager without Node.js needing to issue a security update.

[styfle]

I don't think most users care about that kind of determinism (otherwise, they would set a specific version either system-wise

What makes you think that users are not setting a specific version? I see many projects using npm i -g [email protected]. In fact, package managers print a warning and tell the user the upgrade command when running an outdated version.

[aduh95]

I don't think most users care about that kind of determinism (otherwise, they would set a specific version either system-wise

What makes you think that users are not setting a specific version? I see many projects using npm i -g [email protected]. In fact, package managers print a warning and tell the user the upgrade command when running an outdated version.

Having a fixed default version would not change anything for those users, since they're setting a specific one anyway, right? What I'm saying is that in my experience, folks expect to get the latest version by default, and they are used to set a specific version instead if/when they want determinism (and they should, that's why we added the packageManager field in the package.json).

[andersk]

This change isn’t giving them the latest version. It’s giving them the latest version as of the time they ran corepack prepare. That could be ages earlier than the time they upgraded Node.js and corepack, since it’s a different tool they could easily forget to run regularly when upgrading everything else.

But the security concern isn’t just that they might get an older version—it’s that skipping the hash protection provided by known-good versions may make it easier for an attacker to maliciously alter the code in transit, since HTTPS can much easier to defeat in environments with intercepting proxy middleboxes and dubious certificate authorities.

[aduh95]

This change isn’t giving them the latest version. It’s giving them the latest version as of the time they ran corepack prepare. That could be ages earlier than the time they upgraded Node.js and corepack, since it’s a different tool they could easily forget to run regularly when upgrading everything else.

I reckon the current setup is not ideal, it's a first implementation and I'd be delighted to see it improve in a way that's more convenient for the users. Contributions would be much welcome, as we don't have a clear plan on how to give a way for users to update without relying on them remembering to run a command manually on a regular basis.

But the security concern isn’t just that they might get an older version—it’s that skipping the hash protection provided by known-good versions may make it easier for an attacker to maliciously alter the code in transit, since HTTPS can much easier to defeat in environments with intercepting proxy middleboxes and dubious certificate authorities.

I think https://github.com/nodejs/corepack/issues/10 would be a better response to this, as it should give us the best of both worlds.