sensu-plugins-graphite icon indicating copy to clipboard operation
sensu-plugins-graphite copied to clipboard

Return the set of metrics that are over desired thresholds rather than just the first one

Open himmat62 opened this issue 9 years ago • 4 comments

I need to generate multiple warnings/critical's in one run of check (for checking the disk-usage of 20 machines) by using check-graphite-data.rb  and send it to mailer handler.

As of now i am getting only single output and mail also working.But my requirement is getting multiple(20 machines ,which matches given threshold values ) warnings/critical's.

I have followed link: https://sensuapp.org/docs/0.21/getting-started-with-checks  for setting alerts. i ran below script to check threshold : ruby check-graphite-data.rb -s localhost:8080 -t 'highestCurrent(internal.dev3.*.1.disk-usage.root.used,6)' -w 180 -c 20000 -a 120

send(type, "#{@value['target']} has passed #{type} threshold (#{@data.last})") if below?(type) || above?(type) So, instead of above method, i tried below method because in above method next loop is not running to get multiple warnings/critical's.: if below?(type) || above?(type) puts "CheckGraphiteData #{type} : #{@value['target']} has passed #{type} threshold (#{@data.last})" end But still issue persists.Please suggest me feasible solution.

himmat62 avatar Apr 15 '16 07:04 himmat62

I have this exact same issue. Trying to figure out a solution. As I read the code it appears to me that the intent is to loop through the returned matches from Graphite and evaluate each of them, but it seems like the only alert I get is from the first match returned by Graphite.

smitchelus avatar Jun 06 '16 20:06 smitchelus

The code I was looking at that made me believe that it might report for each match is:

  results = proxy.retrieve_data!
  results.each_pair do |_key, value|
    @value = value
    @data = value['data']
    check_age || check(:critical) || check(:warning)
  end

Followed by:

def check(type) if config[type] send(type, "#{@value['target']} has passed #{type} threshold (#{@data.last})") if below?(type) || above?(type) end end

I haven't had time to add any logging to see why this doesn't work though. I wonder if that send() method is throwing an exception or something which breaks the loop.

smitchelus avatar Jun 06 '16 21:06 smitchelus

We could change this logic to loop over them and append them to an array or hash and then make a check after evaluating each of them which is a common approach in other plugins.

majormoses avatar Jun 10 '17 20:06 majormoses

Would you be willing to attempt a pr for this?

majormoses avatar Jun 10 '17 20:06 majormoses