nethermind icon indicating copy to clipboard operation
nethermind copied to clipboard

Enhance .NET-related monitoring with more counters

Open kkokosa opened this issue 3 years ago • 0 comments

Currently if metrics are enabled, Nethermind.Runner exposes only some .NET runtime-related data, where only two are GC-related: dotnet_total_memory_bytes and dotnet_collection_count_total. This is because the default prometheus-net .NET stats exporter. And it does so by utilizing some APIs like GC or Process class.

We'd like to add an additional set of metrics exported via prometheus-net, at least the ones that are available via EventCounters (also visible f.e. in dotnet-counters tool). At the first stage we mostly need GC-related ones:

% Time in GC since last GC (%) 
Allocation Rate (B / 1 sec)
GC Committed Bytes (MB)
GC Fragmentation (%)
GC Heap Size (MB)
Gen 0 GC Count (Count / 1 sec)
Gen 0 Size (B)
Gen 1 GC Count (Count / 1 sec)
Gen 1 Size (B)
Gen 2 GC Count (Count / 1 sec)
Gen 2 Size (B)
LOH Size (B)
POH (Pinned Object Heap) Size (B)

The solution

An additional Metric class needs to be created (like Nethermind.AccountAbstraction.Metrics for example) that will be later on consumed by the Nethermind.Monitoring.MonitoringService. It should read counters data in-proc.

Additional context

In future we'd like also to export some metrics taken from events (also using in-proc approach). So, maybe we could create it all as a separate "monitoring plugin" from the very beginning?

PS. we may look at Chris .NET Core Counters internals: how to integrate counters in your monitoring pipeline article as an inspiration.

kkokosa avatar Aug 10 '22 13:08 kkokosa