Implement alerting

[cfl0ws]

1 Availability

• [x] 1.1 Server available and reachable - Alert if unreachable
• [x] 1.2 Gaiad running - Alert if Gaiad stops
• [x] 1.3 Validator active, i.e. not jailed - Alert if validator's jailed

2 Performance

• [x] 2.1. Disk utilization - Alert if 70% is exceeded
• [x] 2.3 CPU utilization - Alert if 70% is exceeded
• [x] 2.4 RAM utilization - Alert if 95% is exceeded

3 Validator performance

• [x] 3.3 Caught-up? - Alert if node is not synced
• [x] 3.4 Validator voting? - Alert if validator not voting
• [x] 3.5 Missed blocks - Alert if block threshold is exceeded, currently 4 blocks for testing purposes

4 Connectivity

• [x] 4.1. Number of peers - Alert if number of peers drops below a threshold, currently 1 for testing purposes

5 Validator details

• [x] 5.8 Voting power - Alert if voting power drops below a threshold, currently 50 for testing purposes

6 Proposals

• [x] 6.1 New proposal - Alert if new proposal is created
• [x] 6.2.2 Proposal voting period and period expiration date - Alerts when proposal enters deposit period, voting period, is passed or rejected
• [x] 6.2.3 Validator voting status - Alert if validator hasn't voted within 24 hours of voting period expiration