Broken links after git clone

[alexhochreiter]

Is this the intended behaviour, that when installing a npm package from my gogs git server, it's beeing cloned into the /tmp directory, then a link is created from the current project's node_modules to the directory from /tmp where everything has been cloned into, then said directory from /tmp is beeing deleted.

I don't understand what is going wrong here, nor why git repositories are beeing cloned into /tmp to stay there, as they will be gone after a quick reboot.

OS: Fedora 27 Version: [email protected] Installed as sudo npm install -g npm-git-install

[tad-lispy]

Hello! Thanks for reporting.

It's not intended and sounds like a bug. The temp directory should only be used during build. Then your dependency should be installed from there into your node_modules.

Can you provide more details? Like your package.json file and output from npm install.

[alexhochreiter]

Hi, thanks for the quick response! package.json of a proof-of-concept project:

{
  "name": "huehue",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "install": "npm-git install",
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "",
  "license": "ISC",
  "dependencies": {
    "rabbit.mq.test": "file:../../../../../tmp/npm-git-1171021-10521-5i2cjy.c3wbk",
    "npm-git-install": "^0.3.0"
  },
  "gitDependencies": {
    "rabbit_mq_test":  "http://192.168.0.230:10080/experimente/rabbit_mq_test"
  }
}

Notice the line "rabbit.mq.test": "file:../../../../../tmp/npm-git-1171021-10521-5i2cjy.c3wbk", was not added by me, also the dependency "npm-git-install": "^0.3.0" is still there as i tried to not install it globally first.

NPM install log of npm install in that project's directory:

0 info it worked if it ends with ok
1 verbose cli [ '/usr/bin/node', '/usr/bin/npm', 'install' ]
2 info using [email protected]
3 info using [email protected]
4 verbose npm-session 871824eb14b28283
5 silly install runPreinstallTopLevelLifecycles
6 silly preinstall [email protected]
7 info lifecycle [email protected]~preinstall: [email protected]
8 silly install loadCurrentTree
9 silly install readLocalPackageData
10 silly install loadIdealTree
11 silly install cloneCurrentTreeToIdealTree
12 silly install loadShrinkwrap
13 silly install loadAllDepsIntoIdealTree
14 silly fetchPackageMetaData error for rabbit.mq.test@file:../../../../../tmp/npm-git-1171021-10521-5i2cjy.c3wbk Could not install from "../../../../../tmp/npm-git-1171021-10521-5i2cjy.c3wbk" as it does not contain a package.json file.
15 verbose stack Error: ENOENT: no such file or directory, open '/tmp/npm-git-1171021-10521-5i2cjy.c3wbk/package.json'
16 verbose cwd /home/alex/work/tmp/git-dep-test
17 verbose Linux 4.13.13-300.fc27.x86_64
18 verbose argv "/usr/bin/node" "/usr/bin/npm" "install"
19 verbose node v8.9.1
20 verbose npm  v5.5.1
21 error code ENOLOCAL
22 error Could not install from "../../../../../tmp/npm-git-1171021-10521-5i2cjy.c3wbk" as it does not contain a package.json file.
23 verbose exit [ 1, true ]

Notice the directory /tmp/npm-git-1171021-10521-5i2cjy.c3wbk does not exist (the project i'm trying to fetch from git really contains a package.json - just to clarify that)

After deleting the package-lock.json file and removing the "automatically" added entries from the package.json file, the git repository is successfully cloned again, yet when lsing the node_modules directory, it only shows a broken link.

[alex@Laptop git-dep-test]$ ls node_modules/ -lh
insgesamt 0
lrwxrwxrwx. 1 alex alex 56 22. Nov 09:50 rabbitmqtest -> ../../../../../../tmp/npm-git-1171022-17377-8jdx65.vw6uc
[alex@Laptop git-dep-test]$

[tad-lispy]

Ok, that's weird. I will try to investigate it further later.

[alexhochreiter]

Okay, thank you very much for investigating then! Just to clarify - because you explicitly stated The temp directory should only be used during build. - is the package.json file of the git repository that i'm cloning supposed to have any certain build mechanics? Because right now it's just a plain, minimal, package.json file with a name, version, main and a dependencies property.

[tad-lispy]

You should not need anything else than install script in your package.json (you already have that). I was referring to build of your dependency, which should be set up by it's developer. The dependency is being built in a temp directory according to it's setup and then installed from there. As a consumer of the dependency you should not have to worry about it.

It seems that the problem is caused by this file:../../.. entry in your package.json. The NPM client changed it's behavior and now it adds entries to package.json when user calls npm install. In previous versions you would have to add --save option, but now it's the default behavior.

The fix should be easy (probably just change this line by adding --no-save option), but we need to make sure it will still work with older versions of NPM client.

edit: wrong line link

[kool79]

@lzrski, checked on MacOS 10.13.2 npm5.6 node 8.9.1 npm-git-install uses temporary folder to clone/checkout/install dependency. But this directory automatically deleted after script finished his task. In log excerpt below you can see (at the end) that temp folder (/private/var/folders/g_/qqtcydc11nz_q_mswtry8wsr0000gq/T/npm-git-118027-34165-5y0pln.u94b7/) was not found after installation (while it was exist before).
Maybe temp package wich you use for temp files works incorrectly on mac.

One more note: When dependency installed into main package then symlink is created in node_modules and package.json gets updated with file:// ../../.... link. Maybe it is required for other OS (win, linux) but on mac symlink should be enouth. But I did not use any keys like-save. So I suppose it is npm issue.

[03:16:16] Using gulpfile /private/var/folders/g_/qqtcydc11nz_q_mswtry8wsr0000gq/T/npm-git-118027-34165-5y0pln.u94b7/gulpfile.js
[03:16:16] Starting 'prepublish'...
[03:16:16] Starting 'clean'...
[03:16:16] Finished 'clean' after 14 ms
[03:16:16] Starting 'transpile'...
[03:16:24] Finished 'transpile' after 7.69 s
[03:16:24] Finished 'prepublish' after 7.71 s
added 1247 packages in 63.326s
npm WARN [email protected] requires a peer of request@^2.34 but none is installed. You must install peer dependencies yourself.

+ [email protected]
added 1517 packages in 44.75s
npm WARN prepublish-on-install As of npm@5, `prepublish` scripts are deprecated.
npm WARN prepublish-on-install Use `prepare` for build steps and `prepublishOnly` for upload-only.
npm WARN prepublish-on-install See the deprecation note in `npm help scripts` for more information.

> [email protected] prepublish /Users/o.kulychok/tt/appium
> gulp prepublish

[03:17:26] Using gulpfile ~/tt/appium/gulpfile.js
[03:17:26] Starting 'prepublish'...
[03:17:26] Starting 'clean'...
[03:17:26] Finished 'clean' after 48 ms
[03:17:26] Starting 'transpile'...
[03:17:28] Finished 'transpile' after 2.17 s
[03:17:28] Starting 'fixShrinkwrap'...
Could not find shrinkwrap; skipping fixing shrinkwrap. (Original error: Cannot find module './npm-shrinkwrap.json')
up to date in 153.965s
ta-mac-01:appium o.kulychok$
ta-mac-01:appium o.kulychok$
ta-mac-01:appium o.kulychok$ npm -v
5.6.0
ta-mac-01:appium o.kulychok$ node --version
v8.9.1
ta-mac-01:appium o.kulychok$ cd /private/var/folders/g_/qqtcydc11nz_q_mswtry8wsr0000gq/T/npm-git-118027-34165-5y0pln.u94b7/
-bash: cd: /private/var/folders/g_/qqtcydc11nz_q_mswtry8wsr0000gq/T/npm-git-118027-34165-5y0pln.u94b7/: No such file or directory```  

[kool79]

just checked on windows: nodejs does not create symlinks on windows when installs another module. So here files are copied from tempopary folder to node_modules/<module-name>. Can we force node to copy modules on macOS also?

[tad-lispy]

Hey! Thanks for engaging with my project. It's been a while and I'm pretty sure the issue is not relevant to any of you (see https://github.com/npm/npm/issues/3055#issuecomment-304698029). Please let me know otherwise. Closing.