npmInstall task very slow due to dependency checking

[jechlin]

Thanks for your work on this first of all.

npmInstall takes a very long time to decide its up-to-date, when compared with just running "npm install", something like 20s vs sub-second.

I attempted to force it to just run "npm install" every time using:

npmInstall.outputs.upToDateWhen { false }

but it's not working. My node_modules dir is about 220M.

Any tips on how to disable uptodate checking or make it faster?

[srs]

Thanks.

I see that you have way more node_modules that what I have ever tested with. What version of gradle are you running?

Right now it's setting the outputs/inputs used for up-to-date check in the after-evaluation phase. So, you are setting it too early and it's overwritten in NpmInstallTask.

I should either try to move the outputs/inputs configuration out of after-evaluation phase or have an optional flag to indicate if it should do a up-to-date check or not.

[jechlin]

Actually I don't have that many, the main thing is babel and the es2015 preset, plus gulp... I was pretty shocked at the amount of code that requires.

I was thinking of just copying the package.json to node_modules/. or something, then using that for inputs and outputs. It never happens that someone modifies or removes files under node_modules, so it's overkill checksumming all that imho. cheers, jamie

[mirceade]

+1

[Hildebrand]

I've got a fairly standard jhipster generated project, that now takes 4+ minutes to run the npmInstall task. This makes that I have to leave this step out for now. Running npm install from the command line finishes in 5 seconds.

I guess a combination of checking changes on the package.json file + existence of the node_modules folder would be a workaround until performance has been improved.

Edit: removing the node_modules folder and re-fetching the dependencies resolved all performance issues...

[rafeememon]

Perhaps a solution to this could be to only consider package.json files in node_modules in the outputs rather than the entire directory (re: this line). I'm not sure if this is true of all versions of npm, but the resolved package.json contains a hash that presumably changes if a different version of a dependency is installed. For example:

{
  "_from": "[email protected]",
  "_id": "[email protected]",
  "_inBundle": false,
  "_integrity": "sha512-Qh35tNbwY8SLFELkN3PCLO16EARV+lgcmNkQnoZXfzAF1ASRpeucZYUwBlBzsRAzTb7KyfBaLQ4/K/DLC6MYeA==",
  "_location": "/react",
  "_phantomChildren": {},
  "_requested": {
    "type": "version",
    "registry": true,
    "raw": "[email protected]",
    "name": "react",
    "escapedName": "react",
    "rawSpec": "16.3.0",
    "saveSpec": null,
    "fetchSpec": "16.3.0"
  },
  "_requiredBy": [
    "/"
  ],
  "_resolved": "https://registry.npmjs.org/react/-/react-16.3.0.tgz",
  "_shasum": "fc5a01c68f91e9b38e92cf83f7b795ebdca8ddff",
  "_spec": "[email protected]",
...