worker crashing

[happilymarrieddad]

@jondubois hey man is there an easy way to track this down?

1|handler  | 1527890188676 - Origin: Worker (PID 8416)
1|handler  |    [Error] SocketProtocolError: Socket hung up
1|handler  |     at SCClientSocket._onSCClose (/var/www/handler/node_modules/socketcluster-client/lib/scclientsocket.js:647:15)
1|handler  |     at SCTransport.<anonymous> (/var/www/handler/node_modules/socketcluster-client/lib/scclientsocket.js:292:12)
1|handler  |     at SCTransport.Emitter.emit (/var/www/handler/node_modules/component-emitter/index.js:133:20)
1|handler  |     at SCTransport._onClose (/var/www/handler/node_modules/socketcluster-client/lib/sctransport.js:212:28)
1|handler  |     at WebSocket.SCTransport.wsSocket.onclose (/var/www/handler/node_modules/socketcluster-client/lib/sctransport.js:66:10)
1|handler  |     at WebSocket.onClose (/var/www/handler/node_modules/socketcluster-client/node_modules/ws/lib/event-target.js:124:16)
1|handler  |     at emitTwo (events.js:126:13)
1|handler  |     at WebSocket.emit (events.js:214:7)
1|handler  |     at WebSocket.emitClose (/var/www/handler/node_modules/socketcluster-client/node_modules/ws/lib/websocket.js:172:10)
1|handler  |     at TLSSocket.socketOnClose (/var/www/handler/node_modules/socketcluster-client/node_modules/ws/lib/websocket.js:774:15)
1|handler  | 1527890188684 - Worker 0 exited - Exit code: ****

I want to figure out what's causing it and stop it. I can't seem to figure out what's causing it. None of our other servers are doing this. Thanks!

[happilymarrieddad]

Basically, if there is a data migration process going on on this worker then that process is interrupted and has to restart... sometimes these processes can take hours so a worker crashing is a pretty bad thing. It will get done eventually but sometimes people are waiting on this data. Thanks!

[happilymarrieddad]

@jondubois I guess my question is why is a socket hanging up an Error... that seems wrong...

[jondubois]

@happilymarrieddad Hmm yes that is strange - This kind of error should only trigger on the socket object and shouldn't cause the worker process to crash. Maybe something in your code is explicitly calling a process.exit(...) or process.kill(...)? Or maybe some error event handler was removed and so the error became uncaught?

The other weird thing is the exit code, did you edit the console output for GitHub or does it actually print Exit code: **** to the console?

[happilymarrieddad]

@jondubois No, I just copied and pasted from my terminal. It's super weird.

[jondubois]

@happilymarrieddad Did you try to search for the string **** throughout your project code and/or node_modules dependencies?

Do you have a custom workerClusterController file? The only way I can reproduce this issue is if I add custom code to the workerClusterController and do something like this.workers[0].emit('exit', '****').

That's the only thing I can think of. If it's not that then maybe it's some strange Node.js issue with exit codes that I don't know about. You can try upgrading Node.js version.