Keyboard interrupt sent to app when launched via ecosystem

[aseques]

What's going wrong?

I would like to use ecosystem.config.yaml to launch a python process instead of the command line (that's what I'm currently doing), I am using watchdog (a tool to monitor directorys an launch commands) I get a stacktrace from python telling that there was a keyboard interrupt only when starting from ecosystem file.

32|watch-b |   File "/home/oper/.local/lib/python3.8/site-packages/watchdog/watchmedo.py", line 616, in main
32|watch-b |     parser.dispatch()
32|watch-b |   File "/usr/lib/python3/dist-packages/argh/helpers.py", line 55, in dispatch
32|watch-b |     return dispatch(self, *args, **kwargs)
32|watch-b |   File "/usr/lib/python3/dist-packages/argh/dispatching.py", line 174, in dispatch
32|watch-b |     for line in lines:
32|watch-b |   File "/usr/lib/python3/dist-packages/argh/dispatching.py", line 277, in _execute_command
32|watch-b |     for line in result:
32|watch-b |   File "/usr/lib/python3/dist-packages/argh/dispatching.py", line 231, in _call
32|watch-b |     result = function(namespace_obj)
32|watch-b |   File "/home/oper/.local/lib/python3.8/site-packages/watchdog/watchmedo.py", line 477, in shell_command
32|watch-b |     observe_with(observer, handler, args.directories, args.recursive)
32|watch-b |   File "/home/oper/.local/lib/python3.8/site-packages/watchdog/watchmedo.py", line 114, in observe_with
32|watch-b |     time.sleep(1)
32|watch-b | KeyboardInterrupt

But I can launch the process normally from command line.

How could we reproduce this issue?

pip install watchdog

Command line (it works) watchmedo shell-command --command='pwd' /tmp/

Ecosystem (KeyboardInterrupt)

apps:
    - script: watchmedo
      interpreter: python3
      name: watch-bancs
      args: shell-command --command='pwd' /tmp/
      log_date_format: YYYY-MM-DD HH:mm:ss

Supporting information

--- PM2 report ----------------------------------------------------------------
Date                 : Mon May 10 2021 09:25:48 GMT+0200 (Central European Summer Time)
===============================================================================
--- Daemon -------------------------------------------------
pm2d version         : 4.5.6
node version         : 12.22.1
node path            : /usr/bin/pm2
argv                 : /usr/bin/node,/usr/lib/node_modules/pm2/lib/Daemon.js
argv0                : node
user                 : oper
uid                  : 1001
gid                  : 1001
uptime               : 5292min
===============================================================================
--- CLI ----------------------------------------------------
local pm2            : 4.5.6
node version         : 12.22.1
node path            : /usr/bin/pm2
argv                 : /usr/bin/node,/usr/bin/pm2,report
argv0                : node
user                 : oper
uid                  : 1001
gid                  : 1001
===============================================================================
--- System info --------------------------------------------
arch                 : x64
platform             : linux
type                 : Linux
cpus                 : Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz
cpus nb              : 4
freemem              : 4516147200
totalmem             : 6234554368
home                 : /home/oper
===============================================================================
--- PM2 list -----------------------------------------------
┌─────┬────────────────┬─────────────┬─────────┬─────────┬──────────┬────────┬──────┬───────────┬──────────┬──────────┬──────────┬──────────┐
│ id  │ name           │ namespace   │ version │ mode    │ pid      │ uptime │ ↺    │ status    │ cpu      │ mem      │ user     │ watching │
├─────┼────────────────┼─────────────┼─────────┼─────────┼──────────┼────────┼──────┼───────────┼──────────┼──────────┼──────────┼──────────┤
│ 32  │ watch-bancs    │ default     │ N/A     │ fork    │ 57337    │ 2s     │ 2    │ online    │ 6.4%     │ 14.6mb   │ oper     │ disabled │
└─────┴────────────────┴─────────────┴─────────┴─────────┴──────────┴────────┴──────┴───────────┴──────────┴──────────┴──────────┴──────────┘
Current process list running is not in sync with saved list. App api-weber differs. Type 'pm2 save' to synchronize.
===============================================================================
--- Daemon logs --------------------------------------------
/home/oper/.pm2/pm2.log last 20 lines:
PM2        | 2021-05-10T09:25:45: PM2 log: Stopping app:watch-bancs id:32
PM2        | 2021-05-10T09:25:45: PM2 log: App [watch-bancs:32] exited with code [0] via signal [SIGINT]
PM2        | 2021-05-10T09:25:45: PM2 log: pid=23197 msg=process killed
PM2        | 2021-05-10T09:25:45: PM2 log: App [watch-bancs:32] starting in -fork mode-
PM2        | 2021-05-10T09:25:45: PM2 log: App [watch-bancs:32] online

[hbazan-pp]

I face the same issue, with a KafkaConsumer.poll and it the process gets restarted due to a KeyboardInterrupt. This is also an ecosystem process, with 28 different python processes listening to kafka topics. And it only happens when running with PM2. There is no indication of what's the origin of the KeyboardInterrupt signal

[Alex-zx]

I face the same issue, with a KafkaConsumer.poll

[Thorn2910]

same here - pm2 causing keyboard interrupts on all python scripts

[hbazan-pp]

Are you running the app directly by PM2 or using systenctl? I was running with systemctl, and the service definition was pointing to a .pid file which wasn't being created, so systemctl restarted the service assuming it crashed. Check if setting the service pid to $HOME/.pm2/pid/app-pm_id.pid (as defined on https://pm2.keymetrics.io/docs/usage/application-declaration/)

[aseques]

@hbazan-pp I am running pm2 directly, the problem is that this only happens with the wachdog script, for some other stuff I'm using there are no issues using an ecosystem file (in the same environment)

[dgoadby]

Did this ever get fixed? I just started to use PM2 and the keyboard interrupt problem is a big headache for me. I am not using watchdog.

A few hours later......

I have an MQTT listener which should run for ever. Until I fixed the PM2 command it ran for a few seconds only with a PM2 log message about a Python KeyboardInterrupt.

After a lot of faffing about and trying "monit" I found a solution to my problem and, maybe, others too.

I created a shell script called "run_temps.sh" to run my python program something like this:

# run the workshop tasmota temperature collection application.
python3 '/home/user1/Applications/Python_Apps/workshop_temps/tasmota_to_influxdb_v1.py' 

echo `date` ': Application Ran' > /home/user1/Applications/Python_Apps/workshop_temps/it-worked.txt

The python3 is to ensure that Python three is run. I had strange Python syntax problems until I did. I never got to the bottom of that problem but this method works every time. It was almost like PM2 was using Python7 - which is still on my PC for some legacy code.

Make the script executable in the usual way and test it using "./run_temps.sh"

Whilst you are in the same directory as the shell script add to PM2 using this command:

pm2 start run_temps.sh </dev/null

Redirecting the input completely stopped the problem. The task can still be started, restarted and stopped as normal. I have seen no side effects at all.

After many hours the log is empty and the application is working. ;-)

[blurskye]

still not resolved....