feat: Initial pnpm support

[chawyehsu]

This PR is going to implement the first-class support for the pnpm package manager.

Thanks to @mikrostew for the well-written https://github.com/volta-cli/rfcs/pull/46, I've been able to implement this feature initially. Most of the critical functionalities have been implemented, except for the global package management. Due to a lack of further details on how to intercept pnpm to relocate its global package installs to fit Volta's current layout.

Implemented

• pnpm shim
• pnpm fetch / install / pin
• pnpm hooks
• per project pnpm manipulation
• volta run with --pnpm and --no-pnpm

Not Implemented

• global package manipulation/management via pnpm

In fact the argument parser for_pnpm() has been introduced to try to intercept pnpm args, but I found that there is no proper way to make pnpm's global installs whose original location is <path to pnpm home>/global/5/node_modules fit Volta's current global package image layout <path to volta home>/tools/image/packages. Hence I intercepted all --global pnpm commands and marked them as unimplemented.

Trackers

RFC: https://github.com/volta-cli/rfcs/pull/46 Tracking issue: https://github.com/volta-cli/volta/issues/737

This is currently a draft PR, please leave your review or comment if you have any ideas to improve the PR to push forward.

Demo

❯ .\target\debug\volta.exe list
⚡️ Currently active tools:

    Node: v16.16.0 (default)
    npm: v8.15.1 (default)
    pnpm: v7.7.1 (default)
    Yarn: v1.22.19 (default)
    Tool binaries available:
        cleancss (default)
        yarn-deduplicate (default)

See options for more detailed reports by running `volta list --help`.
DESKTOP in volta on  feature/pnpm [?] is 📦 v1.0.8 via 🦀 v1.62.0
❯ .\target\debug\pnpm.exe -h
Version 7.7.1
Usage: pnpm [command] [flags]
       pnpm [ -h | --help | -v | --version ]

Manage your dependencies:
      add                  Installs a package and any packages that it depends on. By default, any new package is installed as a prod dependency
      import               Generates a pnpm-lock.yaml from an npm package-lock.json (or npm-shrinkwrap.json) file
   i, install              Install all dependencies for a project
  it, install-test         Runs a pnpm install followed immediately by a pnpm test
  ln, link                 Connect the local project to another one
      prune                Removes extraneous packages
  rb, rebuild              Rebuild a package
  rm, remove               Removes packages from node_modules and from the project's package.json
      unlink               Unlinks a package. Like yarn unlink but pnpm re-installs the dependency after removing the external link
  up, update               Updates packages to their latest version based on the specified range

Review your dependencies:
      audit                Checks for known security issues with the installed packages
  ls, list                 Print all the versions of packages that are installed, as well as their dependencies, in a tree-structure
      outdated             Check for outdated packages

Run your scripts:
      exec                 Executes a shell command in scope of a project
      run                  Runs a defined package script
      start                Runs an arbitrary command specified in the package's "start" property of its "scripts" object
   t, test                 Runs a package's "test" script, if one was provided

Other:
      pack
      publish              Publishes a package to the registry
      root

Manage your store:
      store add            Adds new packages to the pnpm store directly. Does not modify any projects or files outside the store
      store prune          Removes unreferenced (extraneous, orphan) packages from the store
      store status         Checks for modified packages in the store

Options:
  -r, --recursive          Run the command for each project in the workspace.
DESKTOP in volta on  feature/pnpm [!] is 📦 v1.1.0-alpha.1 via 🦀 v1.62.0
❯ .\target\debug\pnpm.exe dlx create-vite vite-project --template vanilla
.../../../.pnpm-store/v3/tmp/dlx-31720   |   +6 +
Packages are hard linked from the content-addressable store to the virtual store.
  Content-addressable store is at: D:\.pnpm-store\v3
  Virtual store is at:             ../../../../../.pnpm-store/v3/tmp/dlx-31720/node_modules/.pnpm
.../../../.pnpm-store/v3/tmp/dlx-31720   | Progress: resolved 6, reused 6, downloaded 0, added 6, done

Scaffolding project in D:\workspace\repos\chawyehsu\volta\vite-project...

Done. Now run:

  cd vite-project
  pnpm install
  pnpm run dev

DESKTOP in volta on  feature/pnpm [!?] is 📦 v1.1.0-alpha.1 via 🦀 v1.62.0 took 5s
❯ cd vite-project
DESKTOP in volta\vite-project on  feature/pnpm [!?] via ⬢ v16.16.0
❯ ..\target\debug\pnpm.exe i
Packages: +14
++++++++++++++
Packages are copied from the content-addressable store to the virtual store.
  Content-addressable store is at: D:\.pnpm-store\v3
  Virtual store is at:             node_modules/.pnpm
node_modules/.pnpm/[email protected]/node_modules/esbuild: Running postinstall script, done in 318ms
Progress: resolved 34, reused 0, downloaded 14, added 14, done

devDependencies:
+ vite 3.0.4
DESKTOP in volta\vite-project on  feature/pnpm [!?] via ⬢ v16.16.0
❯ ..\target\debug\volta.exe pin node pnpm
success: pinned [email protected] (with [email protected]) in package.json
success: pinned [email protected] in package.json
DESKTOP in volta\vite-project on  feature/pnpm [!?] via ⬢ v16.16.0 took 3s
❯ cat .\package.json
{
  "name": "vite-project",
  "private": true,
  "version": "0.0.0",
  "type": "module",
  "scripts": {
    "dev": "vite",
    "build": "vite build",
    "preview": "vite preview"
  },
  "devDependencies": {
    "vite": "^3.0.0"
  },
  "volta": {
    "node": "16.16.0",
    "pnpm": "7.7.1"
  }
}
DESKTOP in volta\vite-project on  feature/pnpm [!?] via ⬢ v16.16.0
❯ ..\target\debug\pnpm.exe --version
7.7.1
DESKTOP in volta\vite-project on  feature/pnpm [!?] via ⬢ v16.16.0
❯

[hustcer]

Thanks for the PR

[hustcer]

Any update on this?

[chriskrycho]

@chawyehsu this is excellent to see! 👏🏼 I approved CI to run. @charlespierce or @mikrostew if either of you has time to look and see what needs to be done for the global intercept, that would be fantastic. 🤩

[chawyehsu]

@chriskrycho Thanks! I fixed tests on Unix according to the CI result.

[chriskrycho]

The GH setting is such that I’ll have to reapprove every time, so I'll try to keep an eye on this to keep rerunning it – sorry about the hassle on that front!

[chawyehsu]

I didn't notice there is a --features mock-network flag to turn on all tests including sandboxed smoke tests. I was only using cargo test --all until I saw the CI arguments.

I'll try to fix them today when I have a couple of hours to figure out.

[ScreamZ]

Any news on this boyz ? Need it

[chawyehsu]

@ScreamZ Sorry no ETA, let's wait for the team back from their main work.

[chawyehsu]

pnpx has been deprecated, hence I didn't implement the shimming for it.

it seems reasonable not to block on pnpm globals and leave that unimplemented for now.

It's easy to unblock global commands by simply commenting out the global commands guard in crates/volta-core/src/run/pnpm.rs, this will leave pnpm global packages out of volta's control, however. Blocking global commands may bring inconvenience while allowing the unimplemented to be executed outside volta's layout may get package migration issues in the future. In fact, for me, I may choose to not block global commands but this causes the UX issue that volta can not list global packages installed by pnpm.

[avaly]

Could we get this PR merged as-is with a disclaimer that pnpm support is experimental for now, until the global commands can be properly implemented?

We'd really like to use this in our workflows for local development and Docker builds, so the missing global commands support is not really impacting us.

[runspired]

I second that: pnpm ships often enough that running volta install pnpm@<version-project-uses> many times a day has become annoying :D

[chawyehsu]

Hi folks, thanks for all your interest, response, and emoji reactions to this. Undoubtedly, I hope we could make this happen as possible, Volta and pnpm are both great tools for me to be involved in the ecosystem, and I'm happy with them.

Therefore I made the PR to try to improve these everyday tooling things a little bit more productive and efficient, with a micro goal of hunting some bounties because of my state of out-of-work. The waiting time was a little longer than I expected, expected two months at most for rolling this out. But it's definitely up to the Volta team, I'm not one of its members hence I ain't able to push it to go further. The team is so busy I guess, let's wait a bit more time and we will see.

If you have a strong demand for it right now, you may build a nightly version based on my branch. It's also the base of the local build that I'm currently using.

[chawyehsu]

Changes are applied.

A side note here: When a user updates to a version of Volta that has first-class pnpm support, if they have installed pnpm (as a package) before they might need to uninstall it manually by npm uninstall -g pnpm in order to make the new pnpm shim be available.

[charlespierce]

Thank you @chawyehsu!

[chawyehsu]

Glad that! CI artifacts are now available to try https://github.com/volta-cli/volta/actions/runs/3383641697

[ScreamZ]

Just tried, seems to work, now we need a way to auto-update volta ahah v1.1.0 released

[smblee]

Also tested the artifact above and it works 🥳 What does the official release cycle look like for this binary to be available?

[PindaPixel]

Does this take pnpm/node version compatibility into account?

[chawyehsu]

Does this take pnpm/node version compatibility into account?

@PindaPixel Didn't have any extra implementation for the node version compatibility, the compatibility check will be passed to pnpm.

[smblee]

Hey @charlespierce just wanted to check on when this PR will be officially released

[michaelhays]

@smblee See this comment; there's no ETA as of now.