v18.5.2 causes CPU overloading

[jianglai]

trafficstars

Describe the bug After upgrading to v18.5.2, running cloudcmd cause hundreds of threads to be spawn and pegs CPU utilization at 100%, making the system to be unresponsive.

I downgraded to v18.5.1 and everything went back to normal.

Desktop (please complete the following information):

• OS: Arch Linux
• Node.js version: v23.9.0

If there is specific diagnostic information needed, I'm happy to provide.

[coderaiser]

Could you please try node v20 or v22? v23 unstable and it will be EOL in a week :)

[jianglai]

I tried Node v22.14.0 and observed the same behavior. With v18.5.2 cloudcmd spawns threads uncontrollably until it exhausted memory/CPU resources on the host. With v18.5.1 everything is fine.

[jianglai]

FWIW I tried v18.5.2 in docker and it worked just fine.

[coderaiser]

Looks like the problem here. Could you please comment this line.

It can be related to thread-it. What require('os').cpus() returns to you?

[jianglai]

Yes, after commenting out threadIt.init() it worked.

require('os').cpus() returns a list of 8 cores:

[
  {
    model: 'Intel(R) Core(TM) i7-4870HQ CPU @ 2.50GHz',
    speed: 798,
    times: {
      user: 11786430,
      nice: 5310,
      sys: 7376610,
      idle: 129220490,
      irq: 930620
    }
  },
  {
    model: 'Intel(R) Core(TM) i7-4870HQ CPU @ 2.50GHz',
    speed: 798,
    times: {
      user: 12641120,
      nice: 9530,
      sys: 7831280,
      idle: 128861310,
      irq: 892990
    }
  },
  {
    model: 'Intel(R) Core(TM) i7-4870HQ CPU @ 2.50GHz',
    speed: 800,
    times: {
      user: 14872740,
      nice: 19730,
      sys: 8733070,
      idle: 125569760,
      irq: 1096000
    }
  },
  {
    model: 'Intel(R) Core(TM) i7-4870HQ CPU @ 2.50GHz',
    speed: 798,
    times: {
      user: 13532220,
      nice: 9980,
      sys: 8416410,
      idle: 127037360,
      irq: 1542980
    }
  },
  {
    model: 'Intel(R) Core(TM) i7-4870HQ CPU @ 2.50GHz',
    speed: 798,
    times: {
      user: 20517510,
      nice: 16440,
      sys: 9278540,
      idle: 118451040,
      irq: 1045670
    }
  },
  {
    model: 'Intel(R) Core(TM) i7-4870HQ CPU @ 2.50GHz',
    speed: 798,
    times: {
      user: 14378600,
      nice: 14000,
      sys: 8431180,
      idle: 125947720,
      irq: 1106080
    }
  },
  {
    model: 'Intel(R) Core(TM) i7-4870HQ CPU @ 2.50GHz',
    speed: 798,
    times: {
      user: 18239080,
      nice: 11950,
      sys: 7997870,
      idle: 123051030,
      irq: 908620
    }
  },
  {
    model: 'Intel(R) Core(TM) i7-4870HQ CPU @ 2.50GHz',
    speed: 798,
    times: {
      user: 14085780,
      nice: 5410,
      sys: 8211960,
      idle: 127408800,
      irq: 945630
    }
  }
]

[coderaiser]

I cannot reproduce this, but you can always set THREAD_IT_COUNT=0 to avoid creating of workers. Or you can use count you want instead of 0, this feature is used for User Menu to speed things up.

[dimitrov-anasoft]

Having the same issue with node LTS (v22.15.0) on Debian 12.10. Setting THREAD_IT_COUNT=0 fixes the issue, anything greater than 0 not really.

[coderaiser]

Could you please try this code is it behave in the similar way?

const threadIt = require(‘thread-it’);
// init workers, depend on os.cpus()
threadIt.init();

const putout = threadIt('putout');
const result = await putout(`const t = 'hello'`);

// when you need to override options use
threadIt('putout', {
    count: 5, // default
    time: 1000, // default
    log: () => {}, // default
});

// terminate workers when no need anymore
threadIt.terminate();

Maybe it is related to linux, I cannot reproduce this on Mac Os and my linux has only one core.

[dimitrov-anasoft]

Not sure what the test does, I've added a 10s sleep, otherwise the program terminates relatively fast. I cannot reproduce the issue with this code. I'm not setting THREAD_IT_COUNT.

[coderaiser]

Looks like the problem is here

[coderaiser]

Are you up for a PR?

[dimitrov-anasoft]

I don't think this is the issue. I've added a console log in thread-it init and this is the output:

# npm start

> [email protected] start
> node index.js

---- initing worker  15
---- initing worker  14
---- initing worker  13
---- initing worker  12
---- initing worker  11
---- initing worker  10
---- initing worker  9
---- initing worker  8
---- initing worker  7
---- initing worker  6
---- initing worker  5
---- initing worker  4
---- initing worker  3
---- initing worker  2
---- initing worker  1
---- initing worker  0

No excessive CPU usage. If I restrict the workers to 2, it also respects the setting:

# THREAD_IT_COUNT=2 npm start

> [email protected] start
> node index.js

---- initing worker  1
---- initing worker  0

Then I've added the same log in cloudcmd's thread-it, and this is what happens:

# THREAD_IT_COUNT=2 cloudcmd --port 8086
---- initing worker  1
---- initing worker  0
url: http://localhost:8086/
---- initing worker  1
---- initing worker  0
---- initing worker  1
---- initing worker  0
---- initing worker  1
---- initing worker  0
---- initing worker  1
---- initing worker  0
---- initing worker  1
---- initing worker  0
---- initing worker  1
---- initing worker  0
---- initing worker  1
---- initing worker  0
---- initing worker  1
---- initing worker  0
^Cclosing cloudcmd...

So init is called repeatedly.

[coderaiser]

@dimitrov-anasoft any ideas how to fix it?

[dimitrov-anasoft]

It has something to do with the way JS modules are imported and used. Not sure though, I can take a look over the weekend.

[coderaiser]

Removed thread-it in latest version of Cloud Commander https://github.com/coderaiser/cloudcmd/releases/tag/v18.6.0

Is it works for you now?

[jianglai]

No longer experiencing issues now. Thanks!