[Feature Request] supply monitor_agent stats via RPC or update without_source to skip type monitor_agent

[ryn9]

Is your feature request related to a problem? Please describe.

A times we need to run fluentd without_source to halt input and aide in backend flushing. When doing so - we do not have a method to pull the stats that are normally provided via monitor_agent.

Describe the solution you'd like

supply monitor_agent stats via RPC or update without_source to skip type monitor_agent

Describe alternatives you've considered

N/A

Additional context

No response

[cosmo0920]

in_monitor_agent can handle HTTP request with its published endpoint. The default is http://localhost:24220/api/plugins.json. ref: https://docs.fluentd.org/monitoring-fluentd/monitoring-rest-api

Or, perhaps, it's useful for using fluent-plugin-prometheus to monitor Fluentd stats: https://docs.fluentd.org/monitoring-fluentd/monitoring-prometheus

[ryn9]

@cosmo0920 - these do not work when running with "without_source"

[ashie]

You mean you want to get stats to see the progress of flushing when running fluentd with --without-source option to flush remaining buffers, don't you? It seems to be surely convenient if it can, and need to modify fluentd to realize it.

https://github.com/fluent/fluentd/blob/b04e8084a887dc8395d49c606f474e33a396997c/lib/fluent/root_agent.rb#L152-L154

[ryn9]

@cosmo0920 @ashie - yes it would be good be to able to get stats (I was suggesting via RPC) when --without-source is applied. Further, if you folks like move forward with my other feature request, https://github.com/fluent/fluentd/issues/3692, it would be good to be able to continue getting stats when turning sources down via API.

[cosmo0920]

Currently, Fluentd agent statistics is working on Input plugin mechanism. This feature seems to be hard to implement on Fluentd Core, I guess.