Metrics.Enabled with no PushGatewayUrl

[MarekM25]

It seems that users are using metrics without PushGateway: https://discord.com/channels/629004402170134531/629004402170134537/1003596635235954779

and they're receiving error message

Connected issue: https://github.com/NethermindEth/nethermind/issues/3475

Let's think about better UX (better messages or better defaults)

[paulhauner]

~~Our setup has Prometheus scraping metrics from Nethermind, rather than Nethermind pushing metrics (as per the issue description).~~

~~I've adopted this configuration:~~

NETHERMIND_METRICSCONFIG_ENABLED=true
NETHERMIND_METRICSCONFIG_INTERVALSECONDS=2147483
NETHERMIND_METRICSCONFIG_EXPOSEPORT=XXXX

~~My understanding is that NETHERMIND_METRICSCONFIG_ENABLED must be set to true so that NETHERMIND_METRICSCONFIG_EXPOSEPORT has an effect.~~

~~This workaround means the error is shown at boot and then once every 24 days.~~

~~Since the interval seems to be converted to milliseconds, I computed the value based on Int32.MaxValue / 1000. AFAIK this is the highest possible interval.~~

[paulhauner]

This workaround means the error is shown at boot and then once every 24 days.

After running this config for a while, I don't think it works. I suspect the NETHERMIND_METRICSCONFIG_INTERVALSECONDS defines how often the metrics are updated internally. Whilst they can still be scraped on a more regular interval, the metrics won't be updated.

I'm lead to believe this since practically all of my metrics are still at 0. I can also see nethermind_discovery_bytes_received 282, which is either the most efficient discovery protocol ever, or my metrics aren't updating.

[paulhauner]

Heyo, I'm wondering if #4519 allows for pull-style Prometheus configurations, please? From my naive reading, #4519 seems to produce a different error message rather than allowing metrics to be scraped without a push gateway.

[0xobjectObject]

Yes, that's exactly what I've been experiencing. Using the default Rocket Pool config, which uses a Prometheus pull configuration. All #4519 seems to do is spamming the logs with errors about not setting a PushGateway.

You can mute it by explicitly setting --Metrics.PushGatewayUrl="", but requiring that for pull users seems like worse UX, not better. Is that the intended behavior going forward? Should all pull users have to explicitly set this?

[LukaszRozmej]

Maybe we should just make Metrics.PushGatewayUrl empty by default.