berry icon indicating copy to clipboard operation
berry copied to clipboard

[Feature] Alternative to Node.js experimental corepack to manage package manager version

Open trivikr opened this issue 1 year ago • 11 comments

  • [ ] I'd be willing to implement this feature (contributing guide)
  • [ ] This feature is important to have in this repository; a contrib plugin wouldn't do

Describe the user story

Yarn recommends using experimental corepack to manage yarn versions https://yarnpkg.com/getting-started/install

At the time of writing, Node.js ships corepack which allows pinning the packageManager field in package.json. It's being used in at least tens of thousands of applications searchable in public code.

There has been asks to make corepack stable since May 2022 https://github.com/nodejs/corepack/issues/104 The PR to enable yarn/pnpm corepack binaries by default in https://github.com/nodejs/node/pull/51886, has moved from most approvals to most declines. There's an open PR to remove corepack too at https://github.com/nodejs/node/pull/51981

I'm a very happy corepack+yarn user. I use it all the yarn modern projects I'm primary author of, like https://github.com/aws/aws-sdk-js-codemod, and have got consensus to use corepack in open source packages I maintain with other folks, like https://github.com/facebook/jscodeshift. I also closely monitor/participate in requests to enable corepack in other projects, like GitHub action to setup node in https://github.com/actions/setup-node/issues/531

I'll very be sad when (and if) Node.js removes corepack in future, and have provided a wishlist to package-maintenance team to reduce the impact on ecosystem in https://github.com/nodejs/package-maintenance/issues/609. If corepack can handle package-maintenance team specification, currently called devEngines, and if that helps to get it stable in Node.js core, that would be awesome.

Describe the solution you'd like

Provide some alternative to Node.js experimental corepack to manage yarn versions.

It can be as simple as recommending yarn users to install corepack from npm instead of using the Node.js provided one. This is feasible if corepack maintainers, many of whom help maintain yarn too, plan to continue developing it in case it's removed from Node.js core.

Or it can be like pnpm https://github.com/pnpm/pnpm/releases/tag/v9.7.0, which released a configuration manage-package-manager-versions to manage it's own versions instead of depending on corepack. It may be nice to have similar configuration for yarn users instead of depending on corepack.

Describe the drawbacks of your solution

  • Install corepack from npm: Dependency on npm to install corepack.
    • The Node.js core decided in March'24 that removing npm is a non-goal with 17 approvals https://github.com/nodejs/node/pull/51951, so this wouldn't a problem.
  • Configuration: Requiring developers to install specific yarn version which manages itself.

Describe alternatives you've considered

  • Following up on https://github.com/nodejs/corepack repo on what the maintainers plan to do there.
  • Following up on devEngines specification in https://github.com/nodejs/package-maintenance/pull/594, and who plans to implement it.
  • Switching to a different package manager, like pnpm v9.7.0, which manages it's own version by referring packageManager field introduced by corepack in existing applications.

trivikr avatar Aug 07 '24 13:08 trivikr

The current alternative is to use yarnPath, but we're discussing adding packageManager support to the yarn npm package, so that users can either use npm install -g corepack or npm install -g yarn and have things work.

Of course the friction is still significantly higher than not having to do any of those two actions, and I hope the Node.js leadership will recognize in time its internal brigading issues on all subjects even tangentially related to Npm Inc.

arcanis avatar Aug 07 '24 14:08 arcanis

The current alternative is to use yarnPath

This usually requires yarn binary to be present in the repository, and many people didn't like the idea of adding a binary to their repository as explained in Yarn 4.0 blog post https://yarnpkg.com/blog/release/4.0#installing-yarn

we're discussing adding packageManager support to the yarn npm package

This would be a great solution, if implemented. Yarn users don't need to have modern versions of yarn to manage yarn versions.

use npm install -g corepack

Although this is the easiest option for everyone looking at the current friction, it may not be worth developing corepack further. Especially if alternative package managers, like https://github.com/pnpm/pnpm/releases/tag/v9.7.0, are deciding to manage their own versions and npm wants to remove itself from corepack in https://github.com/nodejs/corepack/pull/418.

trivikr avatar Aug 07 '24 15:08 trivikr

Socket Security wrote a blog post summarizing decision from Node.js PMWG (Package Maintenance Working Group)

https://socket.dev/blog/node-js-takes-steps-towards-removing-corepack

trivikr avatar Aug 08 '24 05:08 trivikr

This usually requires yarn binary to be present in the repository, and many people didn't like the idea of adding a binary to their repository as explained in Yarn 4.0 blog post https://yarnpkg.com/blog/release/4.0#installing-yarn

I would like to add my grain of salt here.

A lot of users never migrated to Yarn v2, as the changes were just too big (PnP by default, corepack, not installed globally anymore) and some people just waited for alternatives to emerge (pnpm).

I think this will be perceived as a regression for a lot of users, but:

  • Migrate away from corepack
  • Make node_modules the default linker (PnP will still be available - just not the default)
  • Remove the need to commit yarn into your repo
  • Make it manage its own version, using packageManager

Could be a great move to (finally) get rid of yarn legacy (v1)

zoontek avatar Aug 19 '24 09:08 zoontek

One concern I have with the suggestion to install corepack from npm is that in some cases users will then also have to manage the corepack version too. Not only making sure to update corepack at times, but sometimes pinning the version being installed in their ci scripts, because there are times when it is necessary to use an end of life nodejs version that the latest corepack has dropped support for. Maybe this is an unavoidable problem, as whatever provides those shims will need this unless it is capable of self-updating. But it worked so nicely with corepack shipped by node, as you knew you would have a compatible version, andt that it would get updated.

A bunch of my libraries are still using 14 in ci simply because there isn't a compelling reason to do a major version bump every year to drop support for the newly end of life release, and doing so is unnecessary friction for users.

Julusian avatar Sep 07 '24 20:09 Julusian

I would like to be able to recommend Yarn (Modern) with good conscience for production use as a migration path from Yarn v1 Classic.

This is the situation I see:

  • Yarn installation without Corepack means using yarnPath and committing a binary. This was already mentioned in the blog section Installing Yarn Oct 23, 2023 with the comment "many people didn't like the idea of adding a binary to their repository, however small.". I don't like this either, when I have to keep updating it every time there is a new version of Yarn released!
  • Yarn installation with Corepack provides a convenient way of installation without requiring committing any binaries. As noted in https://yarnpkg.com/corepack Corepack is "preferred", however it is also declared "experimental":
    • Corepack is currently distributed as Node.js Corepack where it is marked with Stability: 1 - Experimental with the explanation "Non-backward compatible changes or removal may occur in any future release. Use of the feature is not recommended in production environments."
    • There are no committed and published plans to take Corepack out of its experimental status or to provide a Corepack equivalent outside of the Node.js ecosystem.

For comparison, pnpm supports installation using Corepack, however it also publishes its packages to the npm registry so it is not reliant on Corepack.

The frozen Yarn v1 Classic package is also published under https://www.npmjs.com/package/yarn.

Yarn Modern (berry) is not published at all to https://registry.npmjs.org/ so this fallback alternative is not available. Please reconsider this strategy decision. I would welcome Yarn Modern availability in the npm registry to break the reliance on Corepack (probably under a different package name to avoid confusion with Yarn v1 Classic).

This doesn't fit exactly the title of this issue "Alternative to Node.js experimental corepack to manage package manager version" so please let me know if I've overlooked any other issue which already discusses this or if it should be submitted as its own separate issue.

MikeMcC399 avatar Feb 23 '25 08:02 MikeMcC399

I'll just note that committing a very, very small unchanging binary to the repo for bootstrapping is popular in the Java ecosystem with build tools like Gradle and Maven. It seems to work pretty well, and since the binary is tiny and never changes, it doesn't really suffer from the issues that you would have with keeping binaries in git.

ericparton avatar Feb 24 '25 15:02 ericparton

Note: this is an unsupported way of installing

For those looking for a quick workaround: npm install -g @yarnpkg/[email protected]

That's all that is needed to be able to run the yarn command. This works totally fine in our setup, that includes yarn workspaces. Hosted on: https://www.npmjs.com/package/@yarnpkg/cli-dist

hubofgitongithub avatar Feb 25 '25 12:02 hubofgitongithub

Hope yarn support devEngines in package.json.

zanminkian avatar Mar 13 '25 08:03 zanminkian

Corepack will be removed from node in the future https://github.com/nodejs/TSC/pull/1697

zoontek avatar Mar 19 '25 22:03 zoontek

Corepack added support for devEngines in https://github.com/nodejs/corepack/pull/643, and released it in v0.32.0.

The npm trends show that when there was an issue with downloading pnpm package manager version from corepack in late Feb 2025, users switched to downloading it from npm and downloads increased to 2M+ per week.

Details npm-downloads-corepack

If corepack continues to be used by the community, yarn can continue recommending it in their docs. The users will just need to globally install corepack from npm.

I've asked corepack maintainers what metrics they're looking for to continue maintaining corepack in https://github.com/nodejs/corepack/issues/681

trivikr avatar Mar 20 '25 03:03 trivikr

Something to consider here is that yarn 1 is still incredibly popular, installs of it have only climbed since it was declared end of life. https://npmtrends.com/yarn

I believe there is a lot of confusion around the different versions, partly because a lot of things like the github actions runners having yarn1 installed, which has caused issues such as https://github.com/actions/setup-node/issues/1357 where the default configuration for the action doesnt work with a yarn4 project anymore.
I find it not uncommon for new projects to be using yarn1, as that is what their yarn init gives and they dont have any reason to suspect that their yarn isnt up to date

So I think that the ideal replacement for corepack would be published on npm under the yarn name, and can hopefully try and push users away from yarn1. And there should be a push to get places like github actions to include the new version, without users needing to do anything manually.

I expect it could be done in a way where it operates like corepack and can load yarn1 when needed, so it wouldnt have to break workflows. Perhaps it could look at the lockfile to determine what version to use if none is specified in the package.json? Or perhaps that behaviour could vary depending on the version of node being used (ie, if using node 26+ then the default is the latest, otherwise yarn1. That will result it in being a breaking change only when updating node), or perhaps include some deprecation warnings in the output

Julusian avatar Sep 11 '25 22:09 Julusian