Feature: Allow Arborist to configure node_modules and manifest locations independently

[0x80]

Is there an existing issue for this?

• [X] I have searched the existing issues

This issue exists in the latest npm version

• [X] I am using the latest npm

Current Behavior

This is not a bug report, but I didn't see a way to do a feature request.

For Arborist to generate a tree based on node_modules and a manifest, it currently expects them both in the same location and outputs its lockfile there. This is problematic for isolate-package, which wants to generate a pruned lockfile based on the node_modules from the monorepo root, plus an adapted manifest living in a different (isolate) directory.

To make things work, I need to temporarily move the node_modules folder into the isolate directory before running Arborist, and then move it back when finished.

Apart from it being an ugly hack, it prevents multiple processes from executing in parallel. This is a problem because in a monorepo it's likely you want to isolate and deploy multiple packages in parallel.

Generating pruned lockfiles for Yarn and PNPM is not so problematic, but for NPM this is currently the only way to do it AFAIK.

Here is a link to the code with the workaround: https://github.com/0x80/isolate-package/blob/main/src/lib/lockfile/helpers/generate-npm-lockfile.ts

Expected Behavior

Make Arborist accept a separate configuration for locations of the node_modules and manifest files. Please? 🙏

Steps To Reproduce

No response

Environment

No response

[ljharb]

npm commands that affect the disk are not safe to run in parallel, regardless. In a monorepo or not, all these things have to be in serial anyways.

[0x80]

@ljharb Each monorepo package that is targeted for isolation gets its own temp and output directory. The sources for the target plus each internal dependency are copied to those directories, and an adapted manifest file is written there before Arborist is executed.

Would you still consider that unsafe?

Unless Arborist mutates the node_modules folder when it runs, I don't see how it would be unsafe to execute in parallel.

[ljharb]

npm's own cache would still be in play.

[0x80]

Ah, right. Do you think the cache mutates even when I'm only calling buildIdealTree and the node_modules folder is expected to be complete?

If it would be possible to configure Arborist to take the manifest + target location via configuration, that would still be an improvement for me nonetheless, because having to move the node_modules directory to and from the target directory, as I do currently, doesn't feel great.