lighthouse icon indicating copy to clipboard operation
lighthouse copied to clipboard

Metric `validator_monitor_beacon_block_total` starts from 1

Open p2p-sergey-y opened this issue 2 years ago • 0 comments

Description

I want to create a table with blocks, proposed by our validators, but I found out that validator_monitor_beacon_block_total starts immediately from 1. This is a problem because it's really hard to detect exactly this first appearance in Prometheus via changes, rate and other functions.

Metric validator_monitor_beacon_block_total starts from 1 and before that event it doesn't exist. image

Currently, I make my query, which consist of two parts:

  • Find first appearance of metric started from 1. Use query like validator_monitor_beacon_block_total{...} unless validator_monitor_beacon_block_total{...} offset 1m) ==1
  • Find all metric changes. Use query like changes(validator_monitor_beacon_block_total{...}[5m]) > 0

Full query looks like this:

1000* min by (validator) (timestamp( (validator_monitor_beacon_block_total{...} unless validator_monitor_beacon_block_total{...} offset 1m) ==1 or changes(validator_monitor_beacon_block_total{...}[5m]) > 0)) 

And this first part for detecting exactly first metric appearance is really annoying and sometimes unstable. It could be a more way easy if this metric just started from 0 and appeared at lighthouse startup: I could use only changes() query.

The core problem: all metrics related to block proposer changes really rare, and every change is significant, but in current approach it's hard to detect first appearance of this metrics in Prometheus.

Version

lighthouse --version
Lighthouse v3.0.0-18c61a5
BLS library: blst-portable
SHA256 hardware acceleration: false
Specs: mainnet (true), minimal (false), gnosis (true)

Present Behaviour

Metric validator_monitor_beacon_block_total only appears after first block proposal and starts from 1

Expected Behaviour

How should the application behave?

Metric validator_monitor_beacon_block_total should start from 0 at startup time

p2p-sergey-y avatar Aug 24 '22 09:08 p2p-sergey-y