Add OCM server metrics to perf ES

[venkataanil]

OCM server consists of cluster service and account manager service. During ocm load testing we monitor metrics from both these services. In this change, kube-burner pulls metrics from ocm prometheus server for both the services and index them in perf ES accordingly.

We are using the same prometheus queries which OCM team is using in their internal grafana dashboards.

Grafana dashboard for clusters service [1] and account manager service [2] in our perf ES [1] https://grafana.apps.observability.perfscale.devcluster.openshift.com/d/uhc-clusters-service/uhc-clusters-service?orgId=1&var-datasource=ocm-uhc-clusters-service&var-uuid=a5ed9dce-d7be-40a7-a196-1a5da349dbec&from=1657606690358&to=1657621572537 [2] https://grafana.apps.observability.perfscale.devcluster.openshift.com/d/uhc-account-manager/uhc-account-manager?orgId=1&from=1657607400000&to=1657621800000

ES docs for clusters service [3] and account manager service [4] in our perf ES [3] https://perf-results-kibana.apps.observability.perfscale.devcluster.openshift.com/app/discover#/?_g=(filters:!(),refreshInterval:(pause:!t,value:0),time:(from:'2022-07-12T06:30:00.000Z',to:'2022-07-12T11:00:00.000Z'))&_a=(columns:!(),filters:!(),index:'9b6b77e0-fe90-11ec-84f5-13fd83e5c2e7',interval:auto,query:(language:kuery,query:'uuid.keyword%20:%20%22a5ed9dce-d7be-40a7-a196-1a5da349dbec%22%20'),sort:!(!(timestamp,desc))) [4] https://perf-results-kibana.apps.observability.perfscale.devcluster.openshift.com/app/discover#/?_g=(filters:!(),refreshInterval:(pause:!t,value:0),time:(from:'2022-07-12T06:30:00.000Z',to:'2022-07-12T11:00:00.000Z'))&_a=(columns:!(),filters:!(),index:c43ced70-fc60-11ec-b5e5-e5f784bcd40c,interval:auto,query:(language:kuery,query:'uuid.keyword%20:%20%22a5ed9dce-d7be-40a7-a196-1a5da349dbec%22%20'),sort:!(!(timestamp,desc)))

Original internal OCM server grafna dashboards (Maintained by OCM team) https://grafana.app-sre.devshift.net/d/uhc-account-manager/uhc-account-manager?orgId=1&from=now-30m&to=now&var-datasource=app-sre-stage-01-prometheus&var-namespace=uhc-integration&var-rds_datasource=app-sre-stage-01-prometheus&var-rds_namespace=uhc-acct-mngr-integration&refresh=30s https://grafana.app-sre.devshift.net/d/uhc-clusters-service/uhc-clusters-service?orgId=1&var-datasource=app-sre-stage-01-prometheus&var-namespace=uhc-stage&var-pod=All

[smalleni]

@mukrishn @chentex why is this still sitting here?

[mukrishn]

Not sure why is it still open @venkataanil any more changes expected ?

[mukrishn]

@venkataanil confirmed that this one got obsolete by airflow nocp code - https://github.com/cloud-bulldozer/airflow-kubernetes/pull/234