[BUG] Lost support submodules

[trofim24]

Current Behavior:

npm version 7 (I tried both 7.0.0 and the last 7.5.6) does not download submodules.

Expected Behavior:

npm version 6.14.11 has support submodules.

Steps To Reproduce:

I created 3 repos to reproduce the problem. The first contains package.json with dependencies to install. The second contains submodule. The third contains only Readme.md

1. git clone https://github.com/trofim24/npm.git
2. npm install If you do this on version 6.14.11 then the node_modules/npm-bug/npm-bug-submodule folder will exist. If you do this on version 7.5.6 then the node_modules/npm-bug/npm-bug-submodule folder will not exist.

Environment:

• OS: Windows 10 2004 or Debian GNU/Linux 9.13
• npm: 6.14.11 and npm: 7.5.6

[trofim24]

Any updates?

[zpv]

Also experiencing this issue and breaks my workflow. I reverted back to 6 in the meantime. Related (old) issue: https://github.com/npm/npm/issues/6700

[trofim24]

Any updates?

[domq]

Curiously enough, using the latest version of npm, (e.g.) npm i -g https://github.com/Level/rocksdb does what we want i.e. pull the deps/rocksdb/rocksdb submodule out of Facebook. But not when "github.com/Level/rocksdb" is in a package.json as a dependency.

[ljharb]

@domq presumably npm install https://github.com/Level/rocksdb would match package.json (global installs are different than local ones)

[domq]

@ljharb I'm not sure what you mean with “match package.json”, in my experiments it's the git behavior (i.e. whether the clone is recursive or not), that differs between the two situations.

[ljharb]

@domq sorry, what i mean is, you said npm install -g does what you want, but npm install (when it's in package.json) does not. I was asking about what npm install https://github.com/Level/rocksdb does, which is a third, potentially different thing.

[domq]

@ljharb oh, I get it now, sorry. The behavior of npm install https://github.com/Level/rocksdb is indeed the same as when the dependency is in a package.json as you guessed, i.e. a nonrecursive clone is used for the build.

[cjbj]

Still not fixed in npm 9.6.7

[DeeDeeG]

This is both a bug and a documentation issue. I can confirm this bug (the loss of submodule/recursive cloning in dependencies) is still true in npm 10.

The documentation claims submodules will be cloned, but submodules are actually not cloned anymore.

If the repository makes use of submodules, those submodules will be cloned as well.

- From: https://docs.npmjs.com/cli/v10/commands/npm-install

Test Results:

• Submodules are cloned (as in, it works as described in the npm install documentation) if you use npm 6 (npm v6.14.18). ✅
• Submodule cloning doesn't happen for (as in, docs are inaccurate for) npm 7 (v7.24.2), npm 8 (v8.19.4), npm 9 (v9.9.0), and npm latest/10 (v10.2.1). ❌

Implications of the bug: This loss of submodule/recursive cloning breaks some packages when used as git dependencies. For example, Atom's superstring library doesn't build properly on Windows with npm 7.x or newer, as a git dependency, due to a missing vendored win-iconv build-time dependency, which is vendored in as a submodule under the superstring repo. This was working as of npm 6.

What to do about it?: I'd love for submodule/recursive cloning for git dependencies to come back in npm latest. It would un-break some dependencies we still rely on. But if this support is not coming back any time soon, I feel like the docs should be updated in the mean-time to not claim submodules will be cloned when they actually won't. Thanks for considering.

[ljharb]

fwiw, I suddenly have this need as well on two different repos - i'm using a submodule to pull in test fixtures, and having npm install ensure the submodules are available would be very helpful.