panic
panic copied to clipboard
Implement MetricStateStore
Technical Story
When receiving an alert from the Alerter the AlertStore performs the following:
- It stores the alert in
Mongo - Stores the state of the alerted metric inside Redis. This is then used by the
API/UIto display the problems in theOverview Dashboard.
Following the SRP (single responsibility principle) the AlertStore should perform one job only, that being of storing the alert in Mongo. Therefore, we need to create another component which continuously checks the values of each metric and compares them to the alertable thresholds/conditions.
In order to do this change we need to perform the following tasks:
- Remove metric state storing logic from the
AlertStore - Remove internal alerts mechanism as it would no longer be needed
- Develop the
MetricsStateStoreby integrating each monitorable in a granular way using the Strategy pattern
Description
The aim of this ticket is to implement the MetricStateStore component.
Resources:
- https://refactoring.guru/design-patterns/strategy/python/example
- https://auth0.com/blog/strategy-design-pattern-in-python/
Requirements
To implement the MetricStateStore you need to perform the following:
- [ ] Create the connections with RabbitMQ such that the
MetricStateStoreis able to receive the transformed data and alerts configs for every monitorable type - [ ] Implement the routing key <-> strategy logic so that the appropriate strategy is executed for the receive msgs' routing key
- [ ] Implement the heartbeat integration logic
- [ ] Update the
StoreManagercomponent to take theMetricStateStorein consideration - [ ] Update the
run_alerter.pyto include theMetricStateStoreconfig queues
Acceptance criteria
Scenario: The MetricStateStore is able to store the monitorables state for every monitorable type