Cacti
Clear pending device down message queue
Hey Guys
so today I had to reboot my main cacti server and i am getting flooded with device down notifications and there is no way that I know of on how to clear them
It would be awsome if either someone could show me how to or if this could be a feature where you can clear the email que thankfully I dont have these setup for emails to go out when a device goes down that would be horrible! in my case 9k+ emails!!
Some things I've tried
- Disable dead host notification -I found a process running /usr/sbin/sendmail -FCronDaemon -i -odi -oem -oi -t -f apache I kill that and the rate of messages slowed right down -disabled thold plugin still coming in -disabled monitor plugin still coming in
Still getting entries in the log showing a device is recovering ....gah
Oh boy. You know, back in 2003 or so, when I first started talking to Ian, these were all things on my mind, like an email queue instead of a bunch of sendmails. Some of those things are still in the back of my head, which shrinks by the day. It all has something to do with Zinfandel, Cabernet, and the rest of the reds. Oh, and there are those real heavy beers (burp).
Some things I've tried
* Disable dead host notification -I found a process running /usr/sbin/sendmail -FCronDaemon -i -odi -oem -oi -t -f apache I kill that and the rate of messages slowed right down -disabled thold plugin still coming in -disabled monitor plugin still coming inStill getting entries in the log showing a device is recovering ....gah
Clear mail queue in sendmail - all emails are stored in /var/spool/mqueue (on FreeBSD, linux has /var/spool/mail, another path has postfix). Delete here all files (named like qf1044dt1q086605 or df1044dt1q086605)
Few days ago I had the same problem but 40K emails
Message queue is good idea. Is it possible to add queue threshold? Stop message delivery when number of message in queue is higher than threshold? My systems are sending max dozens of emails in one poller cycle. Sometimes any mysql, router, ... issue causes thousand emails. In this case I need stop sending.
Yea, log that as a feature request. I'll add it to the 'Notification' topic for the 1.3 delivery.
@xmacan So looks like I ended up in the same boat today we started taking in events for UP/Down I had it disabled for a long while now getting slammed with events :( I tried your suggestion but I don't find anything in the mailq
Cacti itself doesn't maintain a list of emails to send currently. That said, there could be outstanding PHP processes to send emails like thold which may be running after service is restored still sending out failure notices.
I found a table in threshold that a tually has a list of down states that does not clear if you turn off dead host notifications while its populated which is the "backlog" I posted it in a different report
On Fri., Feb. 5, 2021, 20:49 Mark Brugnoli-Vinten, [email protected] wrote:
Cacti itself doesn't maintain a list of emails to send currently. That said, there could be outstanding PHP processes to send emails like thold which may be running after service is restored still sending out failure notices.
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/Cacti/cacti/issues/3377#issuecomment-774375775, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADGEXTBTF5CZF3PC4S32QL3S5SN4LANCNFSM4LTWKPKQ .
I reported it here
https://github.com/Cacti/plugin_thold/issues/487
It has all my outputs and findings
On Fri., Feb. 5, 2021, 20:52 Sean Mancini, [email protected] wrote:
I found a table in threshold that a tually has a list of down states that does not clear if you turn off dead host notifications while its populated which is the "backlog" I posted it in a different report
On Fri., Feb. 5, 2021, 20:49 Mark Brugnoli-Vinten, < [email protected]> wrote:
Cacti itself doesn't maintain a list of emails to send currently. That said, there could be outstanding PHP processes to send emails like thold which may be running after service is restored still sending out failure notices.
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/Cacti/cacti/issues/3377#issuecomment-774375775, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADGEXTBTF5CZF3PC4S32QL3S5SN4LANCNFSM4LTWKPKQ .