Production Usage / Performance

[Marak]

Hello. Great little library!

I'm evaluating solutions for doing this kind of dependency version management for a large application with thousands of dependencies.

Do you have any comments or specific concerns about using npm-install-version at this scale? Are you aware of any potential areas or use-cases that will cause issues?

Thanks for taking the time to read.

[scott113341]

Hi @Marak, that is certainly an interesting use case! Currently, I do not think this library is ready for production usage on that scale:

1. Its behavior with "more exotic" packages is untested right now. I wouldn't be surprised if there were issues with packages that compile things with node-gyp or have more involved postinstall behavior
2. It is currently synchronous, and can only install one package at a time. If you have to do a lot of installs, with thousands of deps, this is likely unacceptable. However, I think this would be easy to improve.
3. As far as I'm aware, this package isn't used in production at any appreciable scale. So in addition to the previous two points, I'm sure there are things I don't know that I don't know!

That being said, I am not opposed to spending time improving this project, so if you have specific metrics that this project could optimize for, I'm all ears!

[Marak]

@scott113341 -

Thanks for the prompt reply.

Its behavior with "more exotic" packages is untested right now. I wouldn't be surprised if there were issues with packages that compile things with node-gyp or have more involved postinstall behavior

Could you elaborate on this slightly? Will custom modules work at all? Are there any known custom modules that work / don't work?

It is currently synchronous, and can only install one package at a time. If you have to do a lot of installs, with thousands of deps, this is likely unacceptable. However, I think this would be easy to improve.

I don't think that will be a huge issue. Most of our installs will happen one at a time. We already are setup to spawn multiple npm binary subprocesses to do synchronous installs, so the setup should be similar.

I'll test out niv and see what I find.

For a specific metric, I'd like to know if there are any areas in the codebase where you are enumerating over a directory looking for files / subdirs. My primary concern is latency of both install and require of package, specifically when dealing with 1000s of deps in the environment. npm has had some issues with this in the past ( edge cases around cache / enumeration of large node_modules / slow response times for install / etc ).

[scott113341]

Could you elaborate on this slightly? Will custom modules work at all? Are there any known custom modules that work / don't work?

What exactly do you mean by "custom modules"? If you mean things like installing from local paths or GitHub URLs, I believe this will work (though untested). What I'm worried about, for example is Package A 1.0.0 has a dependency on native Library X 12.0.0, but Package B 2.0.0 has a dependency on native Library X 14.0.0. This isn't necessary a unique problem to niv, but it could still be frustrating.

Another example would be Library B 1.0.0 using a node API present in node 0.12 that has been deprecated and removed in node 6.4. It might be impossible to run Library B 1.0.0 alongside, say, Library B 5.0.0.

I'll test out niv and see what I find.

Let me know what you think! I'm happy to spend time improving this tool for a more rigorous use case.

For a specific metric, I'd like to know if there are any areas in the codebase where you are enumerating over a directory looking for files / subdirs.

This does not happen ever!

[scott113341]

#15

[Marak]

@scott113341 - I'm still evaluating solutions here and haven't had much time to actually dig into npm-install-version much

One last question ( for now ), is this utilizing any of npm version 3 features where multiple package versions are stored in the global .npm folder? Would it make sense to try and utilize this existing directory structure?

Thanks again for your time.

[scott113341]

niv doesn't explicitly use the global npm cache, but it installs packages with a regular npm install command, so it may be taking advantage of that caching indirectly.