samza icon indicating copy to clipboard operation
samza copied to clipboard

SAMZA-2412 : Adding SamzaContainer metric that is 0 when container-starting, 1 when started.

Open rmatharu-zz opened this issue 5 years ago • 3 comments

Adding metric that is 0 when container-starting, 1 when started.

Symptom: External monitoring and control (e.g., autosizing) require a way to know when a container has started (e.g., state-restore has completed). Currently, they have to rely on restore-time or container-startup-time metrics, which are a bit tricky to use.

Cause: None

Fix: Added a new metric, a gauge, which is 0 and remains at 0 after the container has started. A Timer on the other hand will go to 0 if its not updated.

Tests: Samza test job with and without state.

rmatharu-zz avatar Dec 02 '19 21:12 rmatharu-zz

You should use a gauge, not counter. In inGraphs, a counter is displayed as a rate (v2-v1/t2-t1).

  1. That's probably not what you want.
  2. Not sure what will happen if v2 < v1.

A gauge should not reset to 0 if not updated, it should retain its last updated value.

prateekm avatar Dec 02 '19 21:12 prateekm

You should use a gauge, not counter. In inGraphs, a counter is displayed as a rate (v2-v1/t2-t1).

  1. That's probably not what you want.
  2. Not sure what will happen if v2 < v1.

A gauge should not reset to 0 if not updated, it should retain its last updated value.

Fair enough, I was confused between Timers and Gauges.

rmatharu-zz avatar Dec 02 '19 22:12 rmatharu-zz

@rmatharu Let's also add this to the metrics docs.

prateekm avatar Dec 03 '19 18:12 prateekm