Hello @lance
First of all thanks for the https://github.com/nodeshift/opossum library which seems to have work pretty well for us so far as a circuit breaker library. 👍
We have also opted to use the https://github.com/nodeshift/opossum-prometheus for monitoring the circuit breakers.
Our nodejs application runs on a cluster mode with a configurable number of worker threads.
- When we tested we found that
opossum-prometheus collects the metrics per worker thread
- Where as in a cluster one would be probably interested in the aggregated metrics of the cluster
- Which means doing something like this https://github.com/siimon/prom-client/blob/master/example/cluster.js
When we brought in the direct dependency of the prom-client library in our application we noticed the data coming back from the prom-client AggregateRegistry::getClusterMetrics() was in coherent and not the aggregated value as expected.
On further debugging we found that the worker threads were responding multiple times to a single message from the cluster master.
The root cause seems to be the multiple dependency on prom-client brought in because of direct dependency of opossum-prometheus & our own dependency to use the AggregateRegistry.
This seems to be a known problem e.g. if you notice this one slanatech/swagger-stats#114 express-prom-bundle have made a fix to the same effect and made prom-client a peer dependency.
A similar approach here would work as well.
I might give it a shot and open a PR for this as well a bit later when I have some time.
Hello @lance
First of all thanks for the https://github.com/nodeshift/opossum library which seems to have work pretty well for us so far as a circuit breaker library. 👍
We have also opted to use the https://github.com/nodeshift/opossum-prometheus for monitoring the circuit breakers.
Our nodejs application runs on a cluster mode with a configurable number of worker threads.
opossum-prometheuscollects the metrics per worker threadWhen we brought in the direct dependency of the
prom-clientlibrary in our application we noticed the data coming back from theprom-clientAggregateRegistry::getClusterMetrics() was in coherent and not the aggregated value as expected.On further debugging we found that the worker threads were responding multiple times to a single message from the cluster master.
The root cause seems to be the multiple dependency on
prom-clientbrought in because of direct dependency ofopossum-prometheus& our own dependency to use the AggregateRegistry.This seems to be a known problem e.g. if you notice this one slanatech/swagger-stats#114 express-prom-bundle have made a fix to the same effect and made
prom-clienta peer dependency.A similar approach here would work as well.
I might give it a shot and open a PR for this as well a bit later when I have some time.