Exclude retention from the scheduler main loop

[mohierf]

Currently the retention store is included in the scheduler main loop and executed as a loop action when the time comes to store the retention (according to Alignak configuration parameter). The retention store may have a quite long duration ... let's say more than one second, creating an overlap in the scheduler loop, see the log hereunder where the retention needed 2.71 seconds.

We may manage the retention in a separate thread to avoid disturbing the main loop ...

[2017-08-06 12:36:39 CEST] INFO: [alignak.scheduler] 3 hosts sent to retention
[2017-08-06 12:36:39 CEST] INFO: [alignak.scheduler] 125 services sent to retention
[2017-08-06 12:36:42 CEST] INFO: [alignak.module.backend_scheduler] 3 hosts saved in retention
[2017-08-06 12:36:42 CEST] INFO: [alignak.module.backend_scheduler] Retention saved in 2.62074494362 seconds
[2017-08-06 12:36:42 CEST] WARNING: [alignak.scheduler] The scheduler loop exceeded the maximum expected loop duration: 1.00. The last loop needed 2.71 seconds to execute. You should update your configuration to reduce the load on this scheduler.

Note: to be discussed ... perharps not necessary if we modify the retention process ?

[mohierf]

At minimum, we may avoid raising a WARNING log when the retention is executed 😉

[ddurieux]

Be carefull, if we run the retention save, the scheduler loop MUST NOT BE RUNNING, because it can changes parameters

[mohierf]

Of course! Do not be afraid, I am the scheduler master :)