pm2
pm2 copied to clipboard
Published
20 hours ago •
Unitech
Unitech
NODE_APP_INSTANCE is always 0 in cluster mode
What's going wrong?
When running in cluster mode, sometimes (not always), NODE_APP_INSTANCE becomes '0' for all the instances in the cluster.
Here is an example:
~$ pm2 status
┌─────┬──────────────────────────┬─────────────┬─────────┬─────────┬──────────┬────────┬──────┬───────────┬──────────┬──────────┬──────────┬──────────┐
│ id │ name │ namespace │ version │ mode │ pid │ uptime │ ↺ │ status │ cpu │ mem │ user │ watching │
├─────┼──────────────────────────┼─────────────┼─────────┼─────────┼──────────┼────────┼──────┼───────────┼──────────┼──────────┼──────────┼──────────┤
│ 0 │ redacted │ default │ N/A │ cluster │ 21397 │ 5m │ 0 │ online │ 0% │ 81.0mb │ abc │ disabled │
│ 2 │ redacted │ default │ N/A │ cluster │ 21411 │ 5m │ 0 │ online │ 0% │ 101.2mb │ abc │ disabled │
│ 4 │ redacted │ default │ N/A │ cluster │ 21424 │ 5m │ 0 │ online │ 0% │ 84.4mb │ abc │ disabled │
...
And then
~$ pm2 env 0
namespace: default
listen_timeout: 10000
kill_timeout: 5000
exp_backoff_restart_delay: 100
max_memory_restart: 4294967296
log_date_format: DD-MM HH:mm:ss.SSS
cwd: /home/abc/current
km_link: false
vizion_running: false
NODE_APP_INSTANCE: 0
PM2_JSON_PROCESSING: true
PM2_USAGE: CLI
NODE_ENV: production
PWD: /home/abc/current
SHELL: /bin/bash
LANG: en_US.UTF-8
INVOCATION_ID: 24c23f0471594fb08fa1b60b109caabc
PATH: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
JOURNAL_STREAM: 9:429352460
LOGNAME: abc
HOME: /home/abc
USER: abc
PM2_HOME: /home/abc/.pm2
redacted: {}
unique_id: 91d61e15-7487-4b24-b71c-1d309d74b90b
_pm2_version: 5.1.0
node_version: 14.15.4
And
~$ pm2 env 2
namespace: default
listen_timeout: 10000
kill_timeout: 5000
exp_backoff_restart_delay: 100
max_memory_restart: 4294967296
log_date_format: DD-MM HH:mm:ss.SSS
cwd: /home/abc/current
km_link: false
vizion_running: false
NODE_APP_INSTANCE: 0
PM2_JSON_PROCESSING: true
PM2_USAGE: CLI
NODE_ENV: production
PWD: /home/abc/current
SHELL: /bin/bash
LANG: en_US.UTF-8
INVOCATION_ID: 24c23f0471594fb08fa1b60b109caabc
PATH: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
JOURNAL_STREAM: 9:429352460
LOGNAME: abc
HOME: /home/abc
USER: abc
PM2_HOME: /home/abc/.pm2
redacted: {}
unique_id: 57076f4f-12a5-469f-81b9-8d3222db9f5c
_pm2_version: 5.1.0
node_version: 14.15.4
And
~$ pm2 env 4
namespace: default
listen_timeout: 10000
kill_timeout: 5000
exp_backoff_restart_delay: 100
max_memory_restart: 4294967296
log_date_format: DD-MM HH:mm:ss.SSS
cwd: /home/abc/current
km_link: false
vizion_running: false
NODE_APP_INSTANCE: 0
PM2_JSON_PROCESSING: true
PM2_USAGE: CLI
NODE_ENV: production
PWD: /home/abc/current
SHELL: /bin/bash
LANG: en_US.UTF-8
INVOCATION_ID: 24c23f0471594fb08fa1b60b109caabc
PATH: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
JOURNAL_STREAM: 9:429352460
LOGNAME: abc
HOME: /home/abc
USER: abc
PM2_HOME: /home/abc/.pm2
redacted: {}
unique_id: 0e6bbe53-56f5-48db-8e91-770abf71e616
_pm2_version: 5.1.0
node_version: 14.15.4
As can be seen, all end up with NODE_APP_INSTANCE '0' making the code to detect instance 0 impossible.
How could we reproduce this issue?
I don't even know how to reproduce it myself, it happens only on some restarts. Once it happens, the only way to fix it is to delete the app completely and start it again from zero.
Supporting information
--- PM2 report ----------------------------------------------------------------
Date : Fri Jul 09 2021 20:33:36 GMT+0200 (Central European Summer Time)
===============================================================================
--- Daemon -------------------------------------------------
pm2d version : 5.1.0
node version : 14.15.4
node path : not found
argv : /usr/bin/node,/usr/lib/node_modules/pm2/lib/Daemon.js
argv0 : node
user : abc
uid : 1001
gid : 1001
uptime : 17min
===============================================================================
--- CLI ----------------------------------------------------
local pm2 : 5.1.0
node version : 14.15.4
node path : /usr/bin/pm2
argv : /usr/bin/node,/usr/bin/pm2,report
argv0 : node
user : abc
uid : 1001
gid : 1001
===============================================================================
--- System info --------------------------------------------
arch : x64
platform : linux
type : Linux
cpus : Intel(R) Xeon(R) CPU D-1531 @ 2.20GHz
cpus nb : 12
freemem : 31975510016
totalmem : 33624117248
home : /home/abc
===============================================================================
--- PM2 list -----------------------------------------------
┌─────┬──────────────────────────┬─────────────┬─────────┬─────────┬──────────┬────────┬──────┬───────────┬──────────┬──────────┬──────────┬──────────┐
│ id │ name │ namespace │ version │ mode │ pid │ uptime │ ↺ │ status │ cpu │ mem │ user │ watching │
├─────┼──────────────────────────┼─────────────┼─────────┼─────────┼──────────┼────────┼──────┼───────────┼──────────┼──────────┼──────────┼──────────┤
│ 0 │ redacted │ default │ N/A │ cluster │ 21397 │ 17m │ 0 │ online │ 1.9% │ 75.7mb │ abc │ disabled │
│ 2 │ redacted │ default │ N/A │ cluster │ 21411 │ 17m │ 0 │ online │ 2.2% │ 73.7mb │ abc │ disabled │
│ 4 │ redacted │ default │ N/A │ cluster │ 21424 │ 17m │ 0 │ online │ 2.1% │ 77.0mb │ abc │ disabled │
...
===============================================================================
--- Daemon logs --------------------------------------------
/home/abc/.pm2/pm2.log last 20 lines:
PM2 | 2021-07-09T20:16:12: PM2 log: App [redacted:2] online
PM2 | 2021-07-09T20:16:12: PM2 log: App [redacted:5] starting in -cluster mode-
PM2 | 2021-07-09T20:16:12: PM2 log: App [redacted:4] online
PM2 | 2021-07-09T20:16:12: PM2 log: App [redacted:6] starting in -cluster mode-
PM2 | 2021-07-09T20:16:12: PM2 log: App [redacted:5] online
PM2 | 2021-07-09T20:16:12: PM2 log: App [redacted:7] starting in -cluster mode-
PM2 | 2021-07-09T20:16:12: PM2 log: App [redacted:6] online
PM2 | 2021-07-09T20:16:12: PM2 log: App [redacted:8] starting in -cluster mode-
PM2 | 2021-07-09T20:16:12: PM2 log: App [redacted:7] online
PM2 | 2021-07-09T20:16:12: PM2 log: App [redacted:9] starting in -cluster mode-
PM2 | 2021-07-09T20:16:12: PM2 log: App [redacted:8] online
PM2 | 2021-07-09T20:16:12: PM2 log: App [redacted:10] starting in -cluster mode-
PM2 | 2021-07-09T20:16:12: PM2 log: App [redacted:9] online
PM2 | 2021-07-09T20:16:12: PM2 log: App [redacted:11] starting in -cluster mode-
PM2 | 2021-07-09T20:16:12: PM2 log: App [redacted:10] online
PM2 | 2021-07-09T20:16:12: PM2 log: App [redacted:11] online
PM2 | 2021-07-09T20:16:12: PM2 log: App [redacted:12] starting in -cluster mode-
PM2 | 2021-07-09T20:16:12: PM2 log: App [redacted:13] starting in -cluster mode-
PM2 | 2021-07-09T20:16:12: PM2 log: App [redacted:13] online
PM2 | 2021-07-09T20:16:12: PM2 log: App [redacted:12] online
Having the same issue, on windows server 2016. Delete and recreate the app is the only workaround for me. Any fixes?
As far as i am concerned PM2 is dead. All issues are ignored, and even support tickets. I requested my money back from the CC company, and got it. They didn't even respond to them.
As far as i am concerned PM2 is dead. All issues are ignored, and even support tickets. I requested my money back from the CC company, and got it. They didn't even respond to them.
correct, unfortunately, pm2 is still the best app for handling node apps in production. If anyone finds something better, please report here so we can all migrate...