metering-operator icon indicating copy to clipboard operation
metering-operator copied to clipboard

Adding Custom query for DB(mariaDB,ES, etc)

Open JooyoungJeong opened this issue 6 years ago • 3 comments

Hi.

Can I Adding Custom query for other DB. Currently, I know that only support queries for prometheus and AWS Billing.

I need to add queries for MariaDB. Is it possible to support other queries?

JooyoungJeong avatar Jan 03 '19 07:01 JooyoungJeong

Today we only support hive for a database backend. Longer term we would like to support other backends for querying and storage, but we're primarily focused on other core reporting features and other integrations for handling upgrades, migrations, etc.

Generally speaking though, adding another DB backend is mostly a matter of the following:

  • Presto needs to support the backend as a connector
  • Need to be able to configure it in Presto's configuration files (done via helm templates)
  • Need to support it as a configuration option in the StorageLocation CRD. That logic is in reporting-operator
  • Need to support exposing the table using a ReportDataSource. That logic is in reporting-operator.

There's a decent amount of work and design needed for us to accomplish that however. It's not something we plan to support in the short-term, but medium-long term I'd expect more backends to be supported. It will also become easier once we've added the first alternative to hive, as many will likely be similar.

chancez avatar Jan 03 '19 17:01 chancez

@chancez

Thank you for your feedback.

My question is that operator-metering can support anything other than Prometheus queries. For example. MariaDB, ElasticSearch, ETC.

Currently, CRD:ReportDataSource uses CRD:ReportPrometheusQuery. Is it possible? ,

  • CRD:ReportDataSource uses CRD:ReportMariaDBQuery.
  • CRD:ReportDataSource uses CRD:ReportElasticSearchQuery.

This means that operator metering can define a new CRD and query raw_data for every interval time.

JooyoungJeong avatar Jan 04 '19 06:01 JooyoungJeong

No, that isn't possible. It wouldn't work quite that way anyways, as I mentioned, Presto will generally be the component making queries when possible, so ideally we never have to write any queries besides regular SQL queries to Presto.

chancez avatar Jan 04 '19 18:01 chancez