cli icon indicating copy to clipboard operation
cli copied to clipboard

[BUG] "sill idealTree buildDeps" takes a lot of time a project with workspaces

Open midudev opened this issue 4 years ago • 16 comments

Is there an existing issue for this?

  • [X] I have searched the existing issues

Current Behavior

After executing npm install the sill idealTree buildDeps step takes a long time (minutes) causing npm@7 working with workspaces slower than npm@6 installing projects separately.

It seems that is fetching packages sequentially instead using parallelization.

Vídeo with the demo:

https://user-images.githubusercontent.com/1561955/118666331-61c26d80-b7f3-11eb-8060-4cb7d28480a9.mp4

Expected Behavior

npm install on npm@7 with workspaces takes less time than npm@6 installing manually each workspace.

Steps To Reproduce

  1. In a environment with a lot of dependencies and workspaces.
  2. With this config:
  "workspaces": [
    "domain",
    "icons",
    "literals"
  ],
  1. Run npm install
  2. See that the step buildDeps take longer than expected.

Some considerations:

  • We're using some npm private packages on our package.json.
  • Workspaces are dependencies of root project as well.

More info: On Travis, builds using npm@6 and installing workspaces manually takes 90s. Using npm@7 it takes 170s using a single npm install.

Environment

  • OS: macOS 11.3.1
  • Node: 16.1.0
  • npm: 7.17.0

midudev avatar May 18 '21 14:05 midudev

I can't provide a example project as it's a commercial project with full of private dependencies but If there's some log or something I could try to give you, feel free to tell me how to extract it.

midudev avatar May 18 '21 14:05 midudev

Tried latest version 7.17.0. Still happening.

midudev avatar Jun 17 '21 17:06 midudev

I am still facing the slowness on latest npm v7 and v8

kumarrishav avatar Nov 02 '21 23:11 kumarrishav

What helped me was to completely uninstall node and then to reinstall it with nvm. However, I dont know if this is going to happen again.

Edit: Okay, it happens again.

vdawg-git avatar Nov 27 '21 10:11 vdawg-git

@v-dawg this is happening in Travis as well. I think it's something related with the way of resolving the tree.

midudev avatar Nov 28 '21 18:11 midudev

Tried with yarn and npm (v8.1.4 and v8.3.0), also the same thing happens. But it's working on 7.19.1

EduApps-CDG avatar Dec 11 '21 07:12 EduApps-CDG

This bug is critical for me. It's so slow that causes ETIMEDOUT even on a empty project.

image

Update:

After running for a while, npm returns:

npm ERR! code ETIMEDOUT
npm ERR! syscall connect
npm ERR! errno ETIMEDOUT
npm ERR! network request to https://registry.npmjs.org/natural failed, reason: connect ETIMEDOUT 2606:4700::6810:1223:443
npm ERR! network This is a problem related to network connectivity.
npm ERR! network In most cases you are behind a proxy or have bad network settings.
npm ERR! network 
npm ERR! network If you are behind a proxy, please make sure that the
npm ERR! network 'proxy' config is set properly.  See: 'npm help config'

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/eduardo/.npm/_logs/2021-12-11T07_56_48_769Z-debug.log

But when I go to https://registry.npmjs.org/natural, it loads pretty fast, so I don't think it's a network problem.

EduApps-CDG avatar Dec 11 '21 07:12 EduApps-CDG

Same issue I think?

npm install
⸨⠂⠂⠂⠂⠂⠂⠂⠂⠂⠂⠂⠂⠂⠂⠂⠂⠂⠂⸩ ⠦ idealTree:myrepo: sill idealTree buildDeps
nvm ls
       v14.17.0
->     v16.11.1
npm --version
8.0.0

Edit, just updated to

⸨#########⠂⠂⠂⠂⠂⠂⠂⠂⠂⸩ ⠙ idealTree:auth0-configuration: timing idealTree:#root Completed in 493084ms

AndrewRayCode avatar Mar 16 '22 21:03 AndrewRayCode

installing react with npx or installing packages for backend :,c

Aborting installation.
  npm install --no-audit --save --save-exact --loglevel error react react-dom react-scripts cra-template has failed.

Deleting generated file... package.json
Deleting frontend/ from M:\nodeB\principal\list-to-task2
Done.
PS M:\nodeB\principal\list-to-task2> mkdir backend
PS M:\nodeB\principal\list-to-task2\backend> npm i dotenv 

added 1 package, and audited 2 packages in 969ms

found 0 vulnerabilities
PS M:\nodeB\principal\list-to-task2\backend> npm i express
PS M:\nodeB\principal\list-to-task2\backend> l idealTree buildDeps 

brayanmf avatar Apr 05 '22 05:04 brayanmf

I tried all the solutions I found on the Internet: removing the proxy, setting the ssl cert to false, removing node_modules, removing the package-lock... nothing worked...

Then, I updated my WSL Ubuntu 20.04 to 22.04 and it worked!

sudo apt update
sudo apt upgrade
sudo apt dist-upgrade
sudo apt install update-manager-core
sudo nano /etc/update-manager/release-upgrades

change the Prompt value to Prompt = lts

sudo do-release-upgrade -d
cat /etc/os-release

for more info see this link

PedroLuisBernardos avatar Apr 15 '22 18:04 PedroLuisBernardos

I tried all the solutions I found on the Internet: removing the proxy, setting the ssl cert to false, removing node_modules, removing the package-lock... nothing worked...

Then, I updated my WSL Ubuntu 20.04 to 22.04 and it worked!

sudo apt update
sudo apt upgrade
sudo apt dist-upgrade
sudo apt install update-manager-core
sudo nano /etc/update-manager/release-upgrades

change the Prompt value to Prompt = lts

sudo do-release-upgrade -d
cat /etc/os-release

for more info see this link

This worked for me. It took a few hours but now everything is working as it should again. Clean slate. Thanks a lot for this @PedroLuisBernardos !!!

sonnystark avatar May 02 '22 14:05 sonnystark

npm set strict-ssl false

works for me

sytandas avatar Aug 07 '22 12:08 sytandas

suddenly encountered this kind of issue again. I had this node app running, then eventually when i run npm i, the installation hangs only.

even if i tried creating new empty project, i tried installing one package, it hangs too.

what seems to be the problem?

i have tried also the solutions provided on stack overflow and on this, still not work.

EmmanDizon avatar Aug 19 '22 13:08 EmmanDizon

suddenly encountered this kind of issue again. I had this node app running, then eventually when i run npm i, the installation hangs only.

even if i tried creating new empty project, i tried installing one package, it hangs too.

what seems to be the problem?

i have tried also the solutions provided on stack overflow and on this, still not work.

What you have tried to resolve that isssue? I am stuck now.

Harshalingole avatar Aug 30 '22 10:08 Harshalingole

I've tried so many ways to run npm install but doesn't work. If you're in a react application, try to use the yarn package running the following command yarn install or just yarn. It worked for me.

And if you're using a corporation network, try to use an wireless signal (Wi-fi).

nicholastn1 avatar Aug 30 '22 14:08 nicholastn1

I tried all the solutions I found on the Internet: removing the proxy, setting the ssl cert to false, removing node_modules, removing the package-lock... nothing worked... Then, I updated my WSL Ubuntu 20.04 to 22.04 and it worked!

sudo apt update
sudo apt upgrade
sudo apt dist-upgrade
sudo apt install update-manager-core
sudo nano /etc/update-manager/release-upgrades

change the Prompt value to Prompt = lts

sudo do-release-upgrade -d
cat /etc/os-release

for more info see this link

This worked for me. It took a few hours but now everything is working as it should again. Clean slate. Thanks a lot for this @PedroLuisBernardos !!!

Yeah you should definitely upgrade to 22.04 I would just set the "Prompt = normal" instead of LTS. Otherwise, the deps will be pretty old.

iMonZ avatar Sep 16 '22 20:09 iMonZ

npm set strict-ssl false

works for me

aiqingr avatar Sep 30 '22 15:09 aiqingr

I can confirm the same issue:

Environment OS: macOS Monterey 12.6 Node: v17.8.0 npm: 8.5.5

abr4xas avatar Oct 18 '22 00:10 abr4xas

I've moved the project to pnpm and now it's using a decent amount of time.

I'm closing the issue as I don't think there's no special interest on this. :)

midudev avatar Oct 19 '22 15:10 midudev

After all the cache clearing and folder deletion, good ol' machine restart done the trick...

guschiavon avatar Nov 03 '22 12:11 guschiavon

I'm having the similar issues. Mine just freeze without any progress. Have anyone found any solution to this yet?

leonism avatar Nov 11 '22 02:11 leonism

For me, it was due to the fact that npm was not failing even if it doesn't get a reply from the server for a very long time, it stays stuck. I figured out it was due to some wrong configuration on my WSL environment that npm requests were ignored silently.

tusharsnx avatar Nov 13 '22 07:11 tusharsnx

npm set strict-ssl false Finally get this and it works!

kiwi6185 avatar Nov 17 '22 15:11 kiwi6185

For me it worked ignoring the .npmrc file

C:\Users\your_user search for .npmrc and rename it as you wish in my case I renamed it like this .npmrc_ and it worked.

AlanneSoares avatar Nov 24 '22 16:11 AlanneSoares

I can confirm the same issue:

Environment OS: macOS Ventura 13.1 Node: v19.3.0 npm: 9.2.0

Nimaw avatar Dec 18 '22 16:12 Nimaw

Same issue.

Environment OS: Ubuntu Linux 22.04 Node: v18.12.1 npm: 8.19.2

Node and NPM installed through NVM.

I have tried everything on this thread without success.

npm set strict-ssl false does not fix the issue.

I have also tried node 16 with same hang.

I cannot downgrade any further than 16 because it breaks min requirements on some packages.

Running npm doctor and there are no reported ping issues so not a connection issue.

haydenyoung avatar Dec 22 '22 23:12 haydenyoung

[Solved!] In my case was I am using a wifi connection with a WPA password (not secure). Changed to my personal 4G (WPA2/3) and now is working again. =/

danpalmieri avatar Dec 26 '22 14:12 danpalmieri

For me it worked to delete the proxy with

npm config delete proxy

and also making sure that registry.npmjs.org is reachable, ie.

ping registry.npmjs.org

rebelmachina avatar Dec 30 '22 05:12 rebelmachina

one of the reason is IPV6.

lospringliu avatar Jan 03 '23 16:01 lospringliu

Same issue

MacOs node: 18.12.0 npm: 8.19.2

Switched to my personal G4 from Wifi and it worked. wtf

devdomsos avatar Jan 05 '23 11:01 devdomsos