Use new custom triggers API to refactor existing playbooks

[aantn]

Today we try to write Robusta actions that can be re-used in multiple scenarios. For example, an action that fetches logs should be usable both in the case of a crashing pod and a prometheus alert.

In the past, we sometimes wrote actions that included triggering logic too. For example, the restart_loop_reporter action is connected to the trigger on_pod_update. This fires very frequently and not only when a pod restarts. Therefore the restart_loop_reporter action has triggering-logic which decides when the action should even do anything.

This breaks the normal separation of triggers and actions. To solve this problem, we introduced the ability to write custom triggers. For example, you can write a crashloop_backoff trigger which inherits from on_pod_update and only fires on pod updates which are due to a crashing pod.

We should rewrite old actions to use the new custom-triggers API. This will lead to more re-usable code.

Actions to rewrite:

• restart_loop_reporter - should be just a logs_enricher action (already exists) and a new on_restart_loop custom trigger
• alert_on_hpa_reached_limit - should be a offer_to_resize_hpa action and a new on_hpa_max custom trigger

[chetak123]

Hey @aantn I'm interested in this issue but having some doubts in it:

1. I was observing restart_loop_reporter file but can't find logs_enricher in there.
2. Also I'm not able to find alert_on_hpa_reached_limit file or any python file related to hpa.

[aantn]

@chetak123 yeah, the logs_enricher is indeed in a different file.

It would probably be easier to start with alert_on_hpa_reached_limit over here.