output of logstash metrics filter does not add up

[biswajit86]

I am trying to understand the output of the logstash metrics filter . I have provided my config below

input {
  beats {    port => 5044  }
}

filter {
   metrics {
       meter => "documents"
       add_tag => "metric"
        flush_interval => 60
   }
}

output {
        if "metric" in [tags] {
                stdout {
                        codec => line { format => "1m rate| %{@timestamp} | %{[documents][rate_1m]} 5m rate: %{[documents][rate_5m]} ( %{[documents][count]} ) "}
                }
        }
        else
        {
                kafka {
                        topic_id => "topbeat-stage"
                        acks => "all"
                }
        }
}

I am generating a meter event every minute. The output from this config looks like this

1m rate| 2016-01-30T03:04:27.086Z | 94.13383390520174 5m rate: 155.7545545326618 ( 17940 )
1m rate| 2016-01-30T03:05:27.087Z | 88.24968300053821 5m rate: 144.52723651339105 ( 22269 )
1m rate| 2016-01-30T03:06:27.086Z | 92.38037875508913 5m rate: 136.8826665572322 ( 26665 )
1m rate| 2016-01-30T03:07:27.086Z | 92.34605776997418 5m rate: 129.6463381559306 ( 31230 )

If I take the last row (1m rate| 2016-01-30T03:07:27.086Z | 92.34605776997418 5m rate: 129.6463381559306 ( 31230 ) ), what does it signify.

• Does it mean, logstash filter plugin is processing ~92 events in the last minute and 129*5 events in the last 5 min
• Is 31230 the total number of events it has processed so far ?
• The difference between the doc count for this row(31230) and the doc count for the previous row(26665) is around ~5k . That does not match either the 1 min rate or the 5 min rate. Am i misinterpreting something ?
• The problem is that if i try to match these numbers to the number of events added in elastic per minute , they do not add up at all.

This issue was originally posted in elastic discuss channel

[andrewvc]

Thanks for posting this, and apologies for the delay @biswajit86 . I actually didn't see this myself in one test, but I'll be taking a second look at it soon.

[smalenfant]

@biswajit86 I'm just starting to look at this plugin for a few things. Glad I came across this as my metrics might be wrong.

My notes from your post

• The count is never reset unless specified. There is a clear_interval setting.
• The rate is per seconds. Is my calculation wrong? (22269-17940)/60 = 72.15.