fluent-plugin-prometheus
fluent-plugin-prometheus copied to clipboard
fluent
Metrics for log throttling per file
Fluentd 1.13 includes a log throttling per file feature. https://github.com/fluent/fluentd/blob/master/CHANGELOG.md#enhancement-2
It would be great if there was a way to publish active throttling through metrics to allow early detection.
@ashie can i pick this up if its of actual use in the tail monitor?
We need to modify in_tail to add such log throttling statics:
https://github.com/fluent/fluentd/blob/64595e2ac291a3e1d2d02feefdc58e623b3abb02/lib/fluent/plugin/in_tail.rb#L208-L211
@ashie for this one which metrics would be useful for us? the number of lines read from each file ?
Oh I see Fine Can I work on this one?
Of course :smiley:
@ashie for this one which metrics would be useful for us? the number of lines read from each file ?
Need to consider but counting number of times throttling is occurred might be down-to-earth approach: https://github.com/fluent/fluentd/blob/64595e2ac291a3e1d2d02feefdc58e623b3abb02/lib/fluent/plugin/in_tail.rb#L1177
@ashie number of times throttling occured means that how many times we are returning a false whenever the check is failed right? Because when i was taking a look at the code, the count of actual number of lines seems to be not possible to be evaluated since we pause,resume and rotate files.
So i will go ahead and count the number of times we hit the check for limit reached