Corepack decisions

[GeoffreyBooth]

Below are the questions I think need resolving in order to wrap up the current Corepack debate.

I gather that some of the members of the TSC are a bit exhausted by all of this; rather than every one of these decisions being handled at the TSC level, another option is to delegate this to the @nodejs/package-maintenance team which has recently suggested chartering with package maintenance being within their scope. We could let them try to address as many of these issues as possible, and either vote or kick back to the TSC the questions that they can’t reach consensus on.

These are the questions/issues that I think need resolution:

• Do we want to allow “placeholder” executables in general? https://github.com/nodejs/node/pull/52107

• If we allow placeholder executables in general, or particularly for package managers:

  • What rules, if any, should govern which projects get placeholder executables?

  • What semver rules would apply to them?

  • How will we handle security vulnerabilities reported in the tools that the placeholders download? (Already addressed by https://github.com/nodejs/node/pull/51917 if we go with the Corepack approach.)

  • Do we want to ship placeholder executables for Yarn and pnpm specifically?

• If we want to ship placeholder executables for Yarn and pnpm, should these executables be:

  • Symlinks to Corepack (https://github.com/nodejs/node/pull/51886)?

  • Download scripts for the referenced tools, possibly including Corepack if that’s the preference of the team maintaining the tool (https://github.com/nodejs/node/issues/51931)?

  • Managed by some other version management tool (asdf, Volta, etc.)?

  • Managed by a new version management tool that we develop that can also manage runtimes (as discussed by the package maintenance team)?

• If we want to ship executables for Yarn and pnpm that are symlinks to Corepack, is Corepack ready to be “enabled by default” to achieve this?

• If we feel that Corepack is not yet ready to be enabled by default and/or marked as stable, what of the following does it need to be considered ready:

  • Does it need a design document and/or documented goals and use cases?

  • Does it need support for the "devEngines" proposal once npm has done so (estimated to happen in April 2024)?

  • What other issues need addressing before it can be enabled by default or marked as stable?

• If we ship executables for Yarn and pnpm that are symlinks to Corepack, what if either of the Yarn or pnpm projects request a change to this arrangement in the future; for example, if the pnpm team desires the pnpm executable stop using Corepack and instead becomes a script to download and run the pnpm standalone installer?

I think this list includes all the concerns raised in https://github.com/nodejs/node/pull/51886, please let me know if I’ve missed anything.

If the decisions for some of the above questions take us in a direction away from enabling Corepack by default, then the question would become what the future of Corepack should be (https://github.com/nodejs/node/pull/51981); but I don’t think we need to contemplate this unless or until that happens.

@nodejs/tsc @nodejs/corepack

[GeoffreyBooth]

It was suggested to me that I add my view on the above so here goes.

• TLDR is that I think we should land https://github.com/nodejs/node/pull/52107 and https://github.com/nodejs/node/pull/51981 and never think about this again.

Placeholder executables in the abstract have numerous downsides as listed in https://github.com/nodejs/node/pull/52107: security risks, release cycles and semver, the need for the TSC to coordinate with whatever external projects we provide placeholders for, the need for the TSC to deal with politics of various projects competing for the bundled treatment. All for the dubious benefit of saving users the need to run an install command, when they still need to approve a prompt to download and install; or the “benefit” to a bundled tool of the appearance of official approval and recommendation from Node.js. The main argument for Corepack seems to be the motivation to promote alternatives to npm, to atone for our sin of bundling npm in the first place and disadvantaging competitors; and to be honest, I just don’t think that that’s something that the project should be doing. I strongly disagree with the argument that since we bundle npm, we’re under some obligation to bundle or otherwise provide alternatives to npm; just, no. We added npm 13 years ago and it’s impractical to remove it now, and if it had never been bundled I doubt we would be discussing adding it now; but it’s not a precedent that justifies anything. If adding npm was a mistake, then adding more external software to our distribution is also a mistake.

And without placeholder executables, then what we have is a bundled Corepack that’s disabled by default; and there’s no benefit from that being bundled in core. Instead of running a command to enable it, users can run a command to download and install it. Corepack itself would benefit from that, as it could have its own release cycle separate from Node’s, and could develop in whatever direction it wants without worrying about blocks from Node collaborators.

[ronag]

The amount of effort this takes relative to its value is, in my opinion, totally out of proportion. Let us vote on whether or not we should even have corepack. If this comes to a vote, I'm of the opinion that we will remove it.

Sorry to be the downer, but based on my perception of conversations, this is the opinion of the quiet/diplomatic majority.

Our inability to come to decisions and continue these discussions forever has already caused us to lose members, and I believe this will continue negatively impacting us.

[aduh95]

@ronag the TSC already decided that we would introduce Yarn through Corepack back in 2021. If you think it’s not worth the effort, you don’t have to put it yourself, and you can choose to trust Corepack maintainers will do the work. If the goal is to not lose members, I doubt that reverting the 2021 decision against the will of the Corepack team will help that, quite the contrary.

[ronag]

the TSC already decided that we would introduce Yarn through Corepack back in 2021

Decisions can be amended. Also, I'm not quite sure that is the whole truth. As far as I know there was never any decision or promise that corepack would be enabled by default per se. Yarn can be bundled in other ways than corepack.

If you think it’s not worth the effort, you don’t have to put it yourself, and you can choose to trust Corepack maintainers will do the work.

The problem is that most of our TSC time is spent endlessly discussing this. Landing corepack without TSC time is not possible as far as I can see. Or do you have a different variation?

trust Corepack maintainers will do the work

Well here lies the issue. I don't trust that they will. The clear lack of interest in all the work @GeoffreyBooth has been putting into this is not a good indication.

[ronag]

If the goal is to not lose members, I doubt that reverting the 2021 decision against the will of the Corepack team will help that, quite the contrary.

I'm actually more concerned in that we come to A decision; whether or not it is a yes or no is important, but not as important. Just continuing with endless discussions is what I'm worrying about in regard to losing members.

[aduh95]

For reference, the 2021 vote details can be found at https://github.com/nodejs/TSC/issues/1012#issuecomment-828776990. I’m bringing it up not to say it can’t be amended, but to point out it’s false to think having a TSC decision would mean that issue won’t be debated again.

FWIW I agree we should find a way to limit the TSC time spent on this issue. But I think there are other options (such as delegating those decisions to a WG). Removing Corepack for this reason would be a very frustrating outcome, and would feel like a governance failure, as it would mean a 3-year effort could be annihilated by a few hostile actors piling up PRs and adding them to the TSC agenda, until enough TSC members got fed up with the discussions going on circles. There are a few vocal members who would want us to remove Corepack, but there are also a quite significant part of our users who are using it, and Corepack is not lacking maintenance, so how would you explain to them Corepack was removed?

[GeoffreyBooth]

Corepack is not lacking maintenance

Maintenance also includes responding to reasonable requests, like the request of those who met in the 2024-01-24 TSC meeting for the Corepack team to define Corepack’s goals: https://github.com/nodejs/node/issues/50963#issuecomment-1908582225. That hasn’t yet been done as far as I’m aware. It’s hard to debate Corepack’s future or potential alternatives when we don’t even know the basics like what problems Corepack aims to solve, and whether there’s consensus that those are goals that the overall Node project shares.

[ronag]

If that moves the issue away from TSC being indecisive, I'm happy with a WG. Though are there sufficient appropriate volunteers?

[aduh95]

Though are there sufficient appropriate volunteers?

I'm hopeful it is the case, I know @wesleytodd has volunteered to lead this effort.

[ShogunPanda]

Here my two cents.

I also agree with @ronag: this topic is totally out of proportion in regards of discussion effort. I think we should cast an hard vote as soon as possible (whatever the outcome is) and then move on.

On the topic it self, I personally believe that node should not carry any executable with it, with the only exception of npm (and thus npx) for historical and onboarding reason. Newcomers will use npm by default and this is totally ok. Nobody says we should provide an alternative (and, FWIW, this usually does not happen in any other ecosystem). Adding additional placeholder executable or technologies like corepack will mostly lead to more security problems and too many environments to deal with (and we already have our own challenges).

Talking about the effort, I don't think that avoiding the user from typing an additional line in the terminal is worth all this. In particular, I think that if I a user wants to use yarn, pnpm or $INSERT_A_NEW_PACKAGE_MANAGER_HERE it means he at least shallowly understands the implication and therefore perfectly capable of typing in the terminal.

I have an alternative idea to address the entire problem but as I don't want to complicate things further, I'll only show it when we cast a decision here.

Before finishing I want to remark that I deeply respect the corepack team effort and I'm not criticising corepack specifically at all. I think it is a good idea but it just does not belong in the Node distribution.

No intent to offend anybody, ever. Peace. ❤️

[GeoffreyBooth]

Besides landing https://github.com/nodejs/node/pull/52107, there’s another “shortcut” to resolving the Corepack debate without tackling each of the questions in the top post: we could just decide that Corepack simply doesn’t belong in core, and remove it from the bundle. We don’t even necessarily need to all agree on the reason or reasons; if the end result is the same, we could just vote to land https://github.com/nodejs/node/pull/51981 and that’s the end of it.

[wesleytodd]

https://github.com/nodejs/node/pull/52107#issuecomment-2018777084

Maybe I should have posted that here? There are a few threads of this discussion and that was the last one mentioning the PMWG proposal to help drive this work I had in my notifications.

[GeoffreyBooth]

I think this can come off the agenda. I think we have a way forward where the package maintenance team is spearheading an effort to rethink what the project wants regarding version management, and as part of that discovery process we should have a proposal for what should happen with Corepack.