rfcs
rfcs copied to clipboard
npm
RFC: Vendoring dependencies
Can you elaborate on what I might be missing here?
Sure.
The motivation for this RFC is my effort to rework how we package npm projects in Nixpkgs -- which presents certain challenges surrounding reproducibility.
The current solution used is to generate large files (warning: 6 MB file) containing dependency trees.
I've proposed an alternate solution which does what I've described as the second alternative -- generating a cacache using a small, purpose-built fetcher, which only relies on a lockfile. (I've done it this over e.g. shelling out to npm to further ensure reproducibility.)
This has proven to work very well for this use case, which is great. But as I said, relying on this is a worse idea than an official solution, since the (historically very low) chances of breakage are there.
We currently use similar solutions for Go, Cargo, and Yarn (v1), utilizing the aforementioned vendoring solutions for the first two, while a home-grown(-ish?) solution for Yarn.
(I'm guessing this comes off as selfish if I'm only doing it for this reason, but I also believe that Go and Cargo must have implemented this for a good reason.)
Vendoring is a resoundingly bad practice that npm should not in any way encourage or ease - there's a reason checking node_modules into version control is similarly discouraged.
Why do you(/others, I'm assuming) believe this? Assuming the vendored dependencies include any and all dependencies that are required for the project, no matter the OS etc, it provides more reproducibility than checking in node_modules.
Reproducibility isn't remotely more important than a manageable git repository or a navigable/comprehensible git diff/log.
Sure, but alternate uses exist, which is evident by Cargo producing a tarball as opposed to a folder.
Is this functionally any different than including all of your dependencies as bundled and running npm pack?
I don't believe so, except for the fact that npm pack depends on the current state of the node_modules folder. Assuming npm ci --ignore-scripts wasn't used for installation, this could introduce impurities depending on what the install/prepare scripts for the dependencies do (e.g. grabbing a prebuilt version of a native dependency).
I think the two serve different purposes, though, and one doesn't require modifying existing projects.
npm pack mirrors as exactly as possible building the tgz file that gets attached during npm publish.
If there is a potential for having things bundled that are missing, this should be solved in the pack/publish process itself so that npm pack becomes as reliable as you need it to be. I don't know what other problem this RFC would be solving, so let's solve that problem itself (if indeed it is determined there is one)
npm packmirrors as exactly as possible building thetgzfile that gets attached duringnpm publish.
Right, but it still depends on the local state of the node_modules folder, no?
Right, but it still depends on the local state of the
node_modulesfolder, no?
Yep. And if that's a problem, then that's the problem to fix.
Looks like I made a pretty dumb factual error here -- both Cargo and Go use folders.
I'll push a revision fixing that, as well as also mentioning npm pack as an alternative.