icinga-powershell-plugins icon indicating copy to clipboard operation
icinga-powershell-plugins copied to clipboard

Strange Value in Icinga Web when Server of is unreachable using Invoke-IcingaCheckTimeSync

Open n-ncls opened this issue 2 years ago • 13 comments

When Invoke-IcingaCheckTimeSync is called on the desired Host via Agent, the Output seems ok:

icinga { Invoke-IcingaCheckTimeSync -Server europe.pool.ntp.org -Warning -10s:10s -Critical -15s:15s -Verbosity 2 }

[UNKNOWN]: Icinga Custom Error was thrown: Custom Exception: ConnectionError

Custom exception occured:
Failed to communicate with the provided server "europe.pool.ntp.org".
A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond

Output in Web Console: grafik

Icinga: 2.13.6 (Master) Agent:

Name                           Value
----                           -----
framework                      1.10.1
mssql                          1.4.0
service                        1.2.0
hyperv                         1.2.0
agent                          2.13.6
inventory                      1.2.0
restapi                        1.2.0
3itplugins                     1.0.4
cluster                        1.2.0
apichecks                      1.2.0
plugins                        1.10.1

n-ncls avatar Jan 18 '23 09:01 n-ncls

Additionally, one suggestion, regarding the same Topic: Change the parameter -Server to accept a list of possible hosts. In this way an alternative NTP Server could be used in case the primary is unreachable. Also more than one IP address could be used, if available, i.e. europe.pool.ntp.org.

Preferred way according to ntp.org, use these 4 Hosts:

0.pool.ntp.org
1.pool.ntp.org
2.pool.ntp.org
3.pool.ntp.org

n-ncls avatar Jan 18 '23 16:01 n-ncls

Has been solved in plugin-version 1.11.0

tectumopticum avatar Sep 15 '23 14:09 tectumopticum

Hello.

the parameter -Server to accept a list of possible hosts. In this way an alternative NTP Server could be used in case the primary is unreachable. Also more than one IP address could be used, if available, i.e. europe.pool.ntp.org.

Please tell me an example of how to correctly use the -Server parameter for several servers. I was unable to use more than 1 NTP server.

But the problem is that if we use a single NTP server and it is temporarily unavailable, then we get UNKNOWN for all Icinga agents.

изображение

Therefore, it is important for us to be able to specify the second and third servers.

Aleksey-Maksimov avatar Jun 17 '24 16:06 Aleksey-Maksimov

And it’s still not clear how the -IgnoreService option works. If we try to use it, the powershell process simply crashes.

Component    Version   Available
---          ---       ---
agent        2.14.2    2.14.2
framework    1.12.3    1.12.3
plugins      1.12.0    1.12.0
service      1.2.0     1.2.0

Aleksey-Maksimov avatar Jun 17 '24 16:06 Aleksey-Maksimov

Hello.

the parameter -Server to accept a list of possible hosts. In this way an alternative NTP Server could be used in case the primary is unreachable. Also more than one IP address could be used, if available, i.e. europe.pool.ntp.org.

Please tell me an example of how to correctly use the -Server parameter for several servers. I was unable to use more than 1 NTP server.

But the problem is that if we use a single NTP server and it is temporarily unavailable, then we get UNKNOWN for all Icinga agents.

изображение

Therefore, it is important for us to be able to specify the second and third servers.

As far as I can see using more than one time-servers is not intended. Your quote references to a proposal of @n-ncls

tectumopticum avatar Jun 18 '24 09:06 tectumopticum

I think that if the authors of the plugin had implemented the ability to poll more than one NTP server, then the problem would have been solved for both the topicstarter and me.

Aleksey-Maksimov avatar Jun 18 '24 09:06 Aleksey-Maksimov

And it’s still not clear how the -IgnoreService option works. If we try to use it, the powershell process simply crashes.

Component    Version   Available
---          ---       ---
agent        2.14.2    2.14.2
framework    1.12.3    1.12.3
plugins      1.12.0    1.12.0
service      1.2.0     1.2.0

Could you send the complete cli-call you're executing? At least "-Server " is mandatory. The PS-instance will close if a parameter is not correctly set or e.g. the time-server-address is unknown or unreachable. To avoid this, you can call something like icinga {Invoke-IcingaCheckTime -Server <server-ip or name> -IgnoreService} This will prevent the closing of the PS and show the stderr.

tectumopticum avatar Jun 18 '24 09:06 tectumopticum

It looks like -IgnoreService has no effect

PS> icinga {Invoke-IcingaCheckTimeSync -Server '10.10.0.14' -Warning -60s:60s -Critical -180s:180s -Verbosity 2 -IgnoreService $true}
Cannot convert value "True" to type "System.Double". Error: "Input string was not in a correct format."
At C:\Program Files\WindowsPowerShell\Modules\icinga-powershell-plugins\compiled\icinga-powershell-plugins.ifw_compilation.psm1:9872 char:12
+     return @{
+            ~~
    + CategoryInfo          : InvalidArgument: (:) [], RuntimeException
    + FullyQualifiedErrorId : InvalidCastFromStringToDoubleOrSingle

[OK] Time Package
\_ [OK] Sync Status: NoLeapWarning
\_ [OK] Time Offset: 0us
\_ [OK] Time Service: Running
| '1010014::ifw_timesync::service'=4;; '1010014::ifw_timesync::offset'=0s;-60:60;-180:180
0

PS> icinga {Invoke-IcingaCheckTimeSync -Server '10.10.0.14' -Warning -60s:60s -Critical -180s:180s -Verbosity 2 -IgnoreService 1}
[OK] Time Package
\_ [OK] Sync Status: NoLeapWarning
\_ [OK] Time Offset: -1s
\_ [OK] Time Service: Running
| '1010014::ifw_timesync::service'=4;; '1010014::ifw_timesync::offset'=-1s;-60:60;-180:180
0

Aleksey-Maksimov avatar Jun 18 '24 09:06 Aleksey-Maksimov

It looks like -IgnoreService has no effect

PS> icinga {Invoke-IcingaCheckTimeSync -Server '10.10.0.14' -Warning -60s:60s -Critical -180s:180s -Verbosity 2 -IgnoreService $true}
Cannot convert value "True" to type "System.Double". Error: "Input string was not in a correct format."
At C:\Program Files\WindowsPowerShell\Modules\icinga-powershell-plugins\compiled\icinga-powershell-plugins.ifw_compilation.psm1:9872 char:12
+     return @{
+            ~~
    + CategoryInfo          : InvalidArgument: (:) [], RuntimeException
    + FullyQualifiedErrorId : InvalidCastFromStringToDoubleOrSingle

[OK] Time Package
\_ [OK] Sync Status: NoLeapWarning
\_ [OK] Time Offset: 0us
\_ [OK] Time Service: Running
| '101600214::ifw_timesync::service'=4;; '101600214::ifw_timesync::offset'=0s;-60:60;-180:180
0
PS> icinga {Invoke-IcingaCheckTimeSync -Server '10.10.0.14' -Warning -60s:60s -Critical -180s:180s -Verbosity 2 -IgnoreService 1}
[OK] Time Package
\_ [OK] Sync Status: NoLeapWarning
\_ [OK] Time Offset: -1s
\_ [OK] Time Service: Running
| '101600214::ifw_timesync::service'=4;; '101600214::ifw_timesync::offset'=-1s;-60:60;-180:180
0

additional arguments after "-IgnoreService" must not be used. So simply try -IgnoreService. Test: stop w32time (Display Name: "Windows Time"), call

Invoke-IcingaCheckTimeSync -Server <your.ntp.ip> -Verbosity 2 -IgnoreService
[OK] Time Package  
\_ [OK] Sync Status: NoLeapWarning  
\_ [OK] Time Offset: 10ms  
\_ [OK] Time Service: Stopped  
| '101210100::ifw_timesync::service'=1;; '101210100::ifw_timesync::offset'=0.01s;;  
0

=> monitor shows still "OK"

"negative test" without "-IgnoreService":

[CRITICAL] Time Package [CRITICAL] Time Service (Stopped)
\_ [OK] Sync Status: NoLeapWarning
\_ [OK] Time Offset: 10ms
\_ [CRITICAL] Time Service: Stopped is not matching threshold Running
| '101210100::ifw_timesync::service'=1;;4 '101210100::ifw_timesync::offset'=0.01s;;
2

tectumopticum avatar Jun 18 '24 10:06 tectumopticum

Yes, now I understand. Thank you for the clarification. But in my opinion, the output of the plugin is not logical. That is, if I specify the -IgnoreService parameter, I expect that there will be no information about the time service in the plugin output. And even more so, I don’t need perfdata about the state of the time service.

Aleksey-Maksimov avatar Jun 18 '24 10:06 Aleksey-Maksimov

We have several specific servers where time synchronization is done by third-party utilities and does not use the NTP protocol. That is, the Windows time service is always turned off on these servers. But using the Invoke-IcingaCheckTimeSync plugin, I want to additionally check the correctness of the time from an additional NTP source. Therefore, I expect to see no information about the Windows Time service when the -IgnoreService option is enabled. In my opinion, this is logical and should even speed up the work of the plugin due to the fact that the request for the status of the time service will be excluded.

Aleksey-Maksimov avatar Jun 18 '24 10:06 Aleksey-Maksimov

Thank you for the input and the discussions. The idea behind this behavior is to ensure that all information and metrics are present at any time. For example you can use the metrics to validate of the time service was actually stopped or running during certain periods of time, which will help troubleshoot issues with the timing.

We have customers who rely on this feature and thats why it is present. You can remove the verbosity part of the output, which then will only print the relevant information in case of errors for the time out-of-sync as example.

LordHepipud avatar Jun 19 '24 10:06 LordHepipud

In this case, how can I exclude the ifw_timesync::service metric that I don’t need?

Aleksey-Maksimov avatar Jun 19 '24 10:06 Aleksey-Maksimov