subql
subql copied to clipboard
subquery
Feature request: alerting capability
Similar to what is proposed by airflow and other ETL frameworks, would be nice if there was a function in the subquery API to raise alerts, and a configuration system to route these alerts to emails or slack...
What would the source of these alerts be? Indexing status of a SubQuery project or something from within the SubQuery project itself?
I was thinking about alerts when the mapping functions have some trouble. Like in the project.yml file, some on_failure_callback configuration that can be pointed to different options like slack or email, and fully managed by Subquery so it doesn't have to run in the sandbox. A global config option would be great already, but can also be made fancy if configurable for each mapping individually, for example to have different mapping functions send failure alerts to different clack channels...
The other use case (the framework itself is having an issue) is interesting too; but I guess it's tricky because you can't rely on something to monitor itself, what if the framework itself is having an issue :) You need then to monitor the monitoring :)
@randombishop we're looking at providing this sort of monitoring of running SubQuery projects in our managed service. It's on our near term roadmap.
In terms of monitoring your own SubQuery project running in your own infra, you could use the logger.error injected methods we have to put this in place
@randombishop In case you run subql-node in your own environment, the subql-node exposes a /metrics api that you can run prometheus to collect data from, and then you can use prometheus alertmanager to trigger alerts.
