rancher-alarms icon indicating copy to clipboard operation
rancher-alarms copied to clipboard

rancher-alarms repeatedly crashing

Open shakefu opened this issue 6 years ago • 3 comments

For whatever reason one of our services is causing rancher-alarms to crash every few minutes. Here's the traceback:

[INFO] 2018-3-23 0:1:29:857 discovered new running service, creating monitor for: ingest/influx
[ERROR] 2018-3-23 0:1:29:898 [TypeError: Cannot read property 'toLowerCase' of undefined] TypeError: Cannot read property 'toLowerCase' of undefined
at initServiceMonitor$ (/usr/src/app/src/server.es6:48:55)
at tryCatch (/usr/src/app/node_modules/babel-core/node_modules/regenerator/runtime.js:61:40)
at GeneratorFunctionPrototype.invoke [as _invoke] (/usr/src/app/node_modules/babel-core/node_modules/regenerator/runtime.js:328:22)
at GeneratorFunctionPrototype.prototype.(anonymous function) [as next] (/usr/src/app/node_modules/babel-core/node_modules/regenerator/runtime.js:94:21)
at invoke (/usr/src/app/node_modules/babel-core/node_modules/regenerator/runtime.js:136:37)
at callInvokeWithMethodAndArg (/usr/src/app/node_modules/babel-core/node_modules/regenerator/runtime.js:172:16)
at previousPromise (/usr/src/app/node_modules/babel-core/node_modules/regenerator/runtime.js:194:19)
at new Promise (/usr/src/app/node_modules/babel-core/node_modules/core-js/modules/es6.promise.js:201:7)
at AsyncIterator.enqueue (/usr/src/app/node_modules/babel-core/node_modules/regenerator/runtime.js:193:13)
at AsyncIterator.prototype.(anonymous function) [as next] (/usr/src/app/node_modules/babel-core/node_modules/regenerator/runtime.js:94:21)
at Object.runtime.async (/usr/src/app/node_modules/babel-core/node_modules/regenerator/runtime.js:215:14)
at initServiceMonitor (/usr/src/app/src/server.es6:58:22)
at updateMonitors$ (/usr/src/app/src/server.es6:101:31)
at tryCatch (/usr/src/app/node_modules/babel-core/node_modules/regenerator/runtime.js:61:40)
at GeneratorFunctionPrototype.invoke [as _invoke] (/usr/src/app/node_modules/babel-core/node_modules/regenerator/runtime.js:328:22)
at GeneratorFunctionPrototype.prototype.(anonymous function) [as next] (/usr/src/app/node_modules/babel-core/node_modules/regenerator/runtime.js:94:21)

At the very least some defensive checking in the method (server.es6 line 48) that grabs the names could make it ignore or alert if somehow the names are missing instead of crashing the service.

shakefu avatar Mar 23 '18 00:03 shakefu

the same, docker-compose:

version: '2' services: rancher-alarms: image: ndelitski/rancher-alarms:v0.1.7 environment: ALARM_SLACK_BOTNAME: rancher-alarms ALARM_SLACK_CHANNEL: '@here' ALARM_SLACK_TEMPLATE: 'Your service #{serviceName} from #{environmentName} become #{healthyState}' ALARM_SLACK_WEBHOOK_URL: https://hooks.slack.com/services/... labels: io.rancher.container.agent.role: environment io.rancher.container.create_agent: 'true'

spender0 avatar Mar 29 '18 12:03 spender0

This is happening to me too. Is there any chance this may get fixed? Or is there a forked version of this project that includes a patch?

dizlexik avatar Oct 09 '18 14:10 dizlexik

So I stumbled onto a fork that has fixed this! And they even have an image on Docker Hub, hooray! https://github.com/eea/rancher-alarms https://hub.docker.com/r/eeacms/rancher-alarms/

dizlexik avatar Oct 09 '18 14:10 dizlexik