opossum-prometheus icon indicating copy to clipboard operation
opossum-prometheus copied to clipboard

Issues due to direct dependency on prom-client library when used for nodejs cluster

Open pp05 opened this issue 3 years ago • 2 comments

Hello @lance First of all thanks for the https://github.com/nodeshift/opossum library which seems to have work pretty well for us so far as a circuit breaker library. 👍 We have also opted to use the https://github.com/nodeshift/opossum-prometheus for monitoring the circuit breakers. Our nodejs application runs on a cluster mode with a configurable number of worker threads.

  • When we tested we found that opossum-prometheus collects the metrics per worker thread
  • Where as in a cluster one would be probably interested in the aggregated metrics of the cluster
  • Which means doing something like this https://github.com/siimon/prom-client/blob/master/example/cluster.js When we brought in the direct dependency of the prom-client library in our application we noticed the data coming back from the prom-client AggregateRegistry::getClusterMetrics() was in coherent and not the aggregated value as expected. On further debugging we found that the worker threads were responding multiple times to a single message from the cluster master.

The root cause seems to be the multiple dependency on prom-client brought in because of direct dependency of opossum-prometheus & our own dependency to use the AggregateRegistry. This seems to be a known problem e.g. if you notice this one https://github.com/slanatech/swagger-stats/issues/114 express-prom-bundle have made a fix to the same effect and made prom-client a peer dependency.

A similar approach here would work as well. I might give it a shot and open a PR for this as well a bit later when I have some time.

pp05 avatar Aug 02 '21 16:08 pp05