Long_Plugin_Output on passive checks ignored

[Mike-Sbg]

When I send the same plugin_output (which contains normal "plugin_output" and "long_plugin_output" everything is putted into the "plugin_output" variable (verified in the status.dat file)

On older Version (eg. Nagios 3) everything works fine

[Mike-Sbg]

The error ist also in an old Nagios Version an is described here (sorry it is in german):

https://monitoring-portal.org/index.php?thread/20919-solved-mehrzeilige-status-informationen-bei-hosts/

[Mike-Sbg]

For reproducing the bug, just enter simply this command:

echo -e "[$(date +%s)] PROCESS_SERVICE_CHECK_RESULT;SERVERNAME;Test;0;Test new entry \n should be long output" >> /var/lib/naemon/naemon.cmd​

Same result from check, triggered as active check, works fine

[nook24]

Hi @Mike-Sbg, AFAIK Naemon parse every line from naemon.cmd until "\n" this should be the reason why it is not working. So naemon process: [$(date +%s)] PROCESS_SERVICE_CHECK_RESULT;SERVERNAME;Test;0;Test new entry \n and should be long output as separated commands if you write it to naemon.cmd.

If you send your command to naemon's query handler (naemon.qh) it should be processed correctly. Example: https://gist.github.com/nook24/5f33431a6cb7f7b4e42f654451119d33

Hope this helps :)

[Mike-Sbg]

Sorry I can't get it to work unix://var/lib/naemon/naemon.qh:0 (No such file or directory) <- but it exsists

... but even if I could, the problem for me is, that I'm force to use the nsca-server and the nsca-ng server, to get the passive check results ...

But the problem isn't specific to naemon ... I also tried it on Nagios 4.0.8 with exactly the same (problem-)result ... the same thing works correctly on nagios 3

[nook24]

Hi @Mike-Sbg, you really need the 3 / (unix:///var/foo...), that was not an typo ;)

But make the long story short, in maybe naemons parsing of naemon.cmd commands is a bit buggy if you try to submit long output... I don't know if this was working in nagios 3.

I had the same problem and the query handler was the only solution to solve this for me.

[Mike-Sbg]

You are right, when I paste it via the naemon.qh (with the 3 slashes ;-) ) everything is parsed correctly ... did you find out, how to use this in den nsca oder nsca-server config?

[nook24]

Nop sorry, I didn't use nsca for years.

[Mike-Sbg]

I was communicating the problem also on the nagios git-hub, and a answer was, that it works in nagios 4.1.1 but not in version 4.0.8 (the official centos/redhat release) ... does anybody know, if this fix will be ported to naemon in near future?

I also found out, that the problem is in the cmd.reader, it separates the input lines hard on the first occurance of an "\n" so the incompleted string is posted to the modul in the checks.c - code.

What I didn't find out for now is, where the cmd-reader is located ... can anybody give me a hint

[Mike-Sbg]

@nook24 If your are interested, I wrote a little parser which reads the input from the nsca(-ng)-Server in pastes it to the .qh- Handler ... it is rather quick&dirty but I works ;-)

[nook24]

@Mike-Sbg I have no use case for this, but may be someone else :)

[jvigna]

Any news about this is there no thinking about fixing this? The BUG is open quit long and seems still to be like this in naemon and nagios alike.

[nook24]

I was facing the exact same issue as i tried to add Checkmk 2.x to my environment. The issue with Checkmk is, that it produce check output like this:

[1629357573] PROCESS_SERVICE_CHECK_RESULT;507cb828-81ca-43fa-92b5-694e96901f19;CPU load;0;15 min load: 0.52 at 2 cores (0.26 per core)\n15 min load: 0.52 at 2 cores (0.26 per core)|load1=0.24;;;0;2 load5=0.35;;;0;2 load15=0.52;;;0;2

According to Nagios https://github.com/NagiosEnterprises/nagioscore/issues/141, a fake string \n should be interpreted as new line internally. Naemon does interpret the string \n as new line, but it will store the result as plugin_output, not split up as plugin_output and long_plugin_output. So you end up with a multi line plugin output.

I was able to workaround this by replaceing the fake \n with a real one str_replace('\n', "\n", $cmd) in php and pass the result to the naemon query handler:

#command run [1629357573] PROCESS_SERVICE_CHECK_RESULT;507cb828-81ca-43fa-92b5-694e96901f19;CPU load;0;15 min load: 0.52 at 2 cores (0.26 per core)
15 min load: 0.52 at 2 cores (0.26 per core)|load1=0.24;;;0;2 load5=0.35;;;0;2 load15=0.52;;;0;2\0

I don't think that this is very handy for most users because it requires to have a fake naemon.cmd to fetch the passive check results, manipulate theme and route theme to the query handler.

[systemdlete]

Sorry to necropost, but I have been dealing with this exact issue recently myself, and maybe found a solution.

There is the spooling option, whereby you can write results to a spool directory and naemon will consume them from there. See https://www.naemon.io/documentation/developer/spoolfolder.html

[nook24]

It is probably a good idea to add a list of all methods to the Naemon docs. Currently the following methods exists, to pass external commands and check results to Naemon. Or at least, that are the methods I’m aware of.

Naemon default methods

External command file naemon.cmd

Pros	Cons
Easy to use	Can get overwritten with a text file
	No feedback
	No long output

Query Handler naemon.qh

Pros	Cons
Support for long output	Complicated to use
Provides feedback (OK / Error)
Error if Naemon is not running

Spoolfolder

Pros	Cons
Easy to use	I had all sort of errors with this such as segmentation faults and bus errors

Through Broker Modules

I use the Statusengine Event Broker to pass all external commands or passive check results to Naemon. How ever, this is basically undocumented at the moment. Good places to look up are:

• https://github.com/statusengine/broker/blob/master/src/MessageHandler/MessageHandler.h#L152
• https://github.com/it-novum/openITCOCKPIT/blob/development/src/itnovum/openITCOCKPIT/Monitoring/Naemon/ExternalCommands.php#L151-L166
• https://github.com/it-novum/openITCOCKPIT/blob/development/src/itnovum/openITCOCKPIT/Monitoring/Naemon/ExternalCommands.php#L484-L494

Mod_Gearman provides a send_gearman binary which can also be used to send check results to Naemon. I personally haven’t used this for a very long time so I’m not sure about if it is able to process long output or not.

[sni]

just my 2 cents...

• using the spoolfolder for decades now and never had any issue. If you experience segfaults with public APIs, please always open an issue.
• send_gearman supports long plugin output

[systemdlete]

I am currently using spoolfolder and have not noticed any problems with it.