incubator-uniffle
incubator-uniffle copied to clipboard
apache
[Feature Request] Support push metrics to Prometheus Pushgateway
The push mode will be more convenient and more real-time. We have implemented it. If necessary, I will create a PR
Why will the push mode be more convenient and more real-time? Could you give some examples?
1.We need install node exporter in every host, and if not, real-time performance cannot be ensured. 2.We need change the configration of node exporter after we add an instance.
In our production environment, we use crontab to fetch the servers' metrics every minute and push the metrics to our prometheus server. I don't get the difference if we use exporter.
This depends on operation and maintenance. If automation tools is well, it may not be inconvenient.
@jerqi Can you share how do you collect metrics in your k8s cluster?
We plan to use a k8s sidecar to collect the metrics.
I think we should support export metrics, like hadoop,hbase etc. Users can implement custom export plugin to export metrics to their monitoring system. WDYT? @jerqi
I think we should support export metrics, like hadoop,hbase etc. Users can implement custom export plugin to export metrics to their monitoring system. WDYT? @jerqi
I'm not familiar with this aspect. If other mature systems have this , I think it will be ok for me. @zuston Do you have some suggestion?
The similar mechanism is supported by other systems, like Flink: https://nightlies.apache.org/flink/flink-docs-release-1.13/docs/deployment/metric_reporters/
I think this is a good improvement of observability
