Too many open files

[dacevedo12]

Using makeNodeJsEnvironment fails due to Too many open files, whereas a normal npm install doesn't.

some of the most popular distros provide fairly limited defaults, so this is likely to have a wide impact

[kamadorueda]

Can you post the log or some reproduction steps please?

[dacevedo12]

ulimit -Sn
1024
ulimit -Hn
1048576

https://gitlab.com/fluidattacks/universe/-/commit/c5a2162c6978fd4456dc9bedb5678daa57980f3c
m . /docs

--- more packages ---
copying path '/nix/store/m11vbfnvllq1injsp5imghfkki0wcvkl-ast-1.11.1.tgz' from 'https://cache.nixos.org'...
copying path '/nix/store/rd5ri9c4w8z4y3jy4rwcfzliql6jn5wv-asap-2.0.6.tgz' from 'https://cache.nixos.org'...
copying path '/nix/store/l00rwxs3myr6h065qgl6akvckw3irgv1-array-flatten-2.1.2.tgz' from 'https://cache.nixos.org'...
copying path '/nix/store/ni360d14qaqyry6cfq62653dgxqy193a-at-least-node-1.0.0.tgz' from 'https://cache.nixos.org'...
error: opening directory '/nix/store/v03lrsk9lz0yk5h13r66vzi7rxm9ksk4-std-env.json': Too many open files

[kamadorueda]

that's a bug with Nix itself, we cannot solve it, only mitigate it, that error is happening while Nix fetches things from the cache, all we could is increasing the limit at runtime in the container, and document the fix so that people running things on their laptop can apply it. Maybe we can also reduce Nix's concurrency, but it's already low

[dacevedo12]

Would the same happen with a large collection of python dependencies or are there differences in the way deps are built due to package manager constraints?

[kamadorueda]

I'd say the same would happen anytime Nix wants to fetch many things from the cache at once. Npm is just once of those cases where you would fetch thousands of store paths from the cache, but a lot of python packages, or even building a big project all at once could trigger the problem

to be honest is hard to tell, it's not a bug in makes, it's a bug in Nix and Nix is written in c++