fluent-bit icon indicating copy to clipboard operation
fluent-bit copied to clipboard

Configure different scraping intervals for individual metrics

Open kubotat opened this issue 2 years ago • 0 comments

Is your feature request related to a problem? Please describe.

With the node_exporter plugin, it is able to capture a variety of metrics like, CPU, Memory, Disk and Network. The scaping interval can be customized by ‘scrape_interval’ option like the following sample configuration.

[INPUT]
    name   node_exporter_metrics
    tag    osmetrics
    scrape_interval    30
    Storage.type    filesystem

In the current implementation, a single scraping interval is applied for all metrics and cannot be configured for individual metrics. Support to alter scrape intervals or disable collection for individual metrics offers users additional flexibility to customize metric collection for their needs.

For instance, metrics such as CPU and Memory could be captured more frequently, while other metrics such as disk and network can be collected less often. Support to set scrape intervals or disable collection for individual metrics helps users to optimize the amount of metrics captured. Disk for instance, has 14 different metrics and shorter intervals can cause high resource utilization at client side and also consume disk space at the backend system like Prometheus and Splunk.

> The list of disk metrics
disk.discarded_sectors_total
disk.discards_completed_total
disk.discards_merged_total
disk.discard_time_seconds_total
disk.io_time_seconds_total
disk.io_time_weighted_seconds_total
disk.read_bytes_total
disk.reads_completed_total
disk.reads_merged_total
disk.read_time_seconds_total
disk.writes_completed_total
disk.writes_merged_total
disk.write_time_seconds_total
disk.written_bytes_total

Describe the solution you'd like

Enhance ‘node_exporter’ plugin to allow users to configure different scraping intervals or disable collection for each individual metric type. Describe alternatives you've considered

Additional context

kubotat avatar Jul 21 '22 22:07 kubotat