Support arithmetic operations on queries

[hexedpackets]

The idea is to add support for user natural language support for arithmetic operations upon timeseries'

E.g. a Grafana user with Queries "A" and "B" could create a new Query "C", defined as "A[rps] / B[rps]". This will result in Series "C" which will be (unsurprisingly): [ A[i] / B[i], A[i+1] / B[i+1], A[i+2] / B[i+2]... ] e.g.

A=[4.0,5.0,2.8,...]
B=[4.0,10.0,2.7,...]
→ 
C=[1.0,0.5,1.037,...]

---

Q4 2020

• The RFC: Add Arithmetic support for queries contains detailed information on this work.
• These "Heroic Arithmetic Support" slides describe the RFC in a more visual fashion and includes Architecture, Request flow and more.
• There is also heroic-benchmark which was authored and used to benchmark the different libraries that support natural-language mathematic expression parsing (namely MxParser and exp4j)

---

Original Content (William) ———————————— This would allow us to perform arithmetic operations on the backend, instead of returning all of the data from the queries and relying on something like Grafana to do it.

PR #393 was opened for this originally, it needs to be rebased and then reviewed. The work lives on the jsferrei/arithmetic-engine branch.

[stale[bot]]

⚠ This issue has been automatically marked as stale because it has not been updated for at least two months. If this is a bug and you can still reproduce this error on the master branch, please reply with all of the information you have about it in order to keep the issue open. This issue will automatically be closed in ten days if no further activity occurs. Thank you for all your contributions.

[philipcristiano]

@dmichel1 mentioned that this would also allow for ratios of metrics. For a use case I'm dealing with I'd love to be able to compare 2 metrics. This would help me out by allowing graphs (and hopefully alerts) where I track a value and the threshold, then show the progress to that threshold. More specifically Kubernetes resource usage compared to requests/limits.

[hexedpackets]

One approach I started experimenting with is extending the batch endpoint with additional reducers that act as aggregations after the original query is assembled. I think its possible to reuse the existing aggregations this way by passing the results of the query back into a new aggregation instance.

In the QueryResource we can transform the data post-query and before the response, so this is a good place to add an additional reducer.

The batch endpoint query would look something like

{
  "queries": {
    "A": {...},
    "B": {...}
  },
  "reducer": {
    "of": ["A", "B"],
    "aggregation": {
      "type": "sum",
      "sampling": {"unit": "seconds", "value": 3600}
    }
   }
}

and then deserialize that into a SumInstance that gets the query results fed into it.

I'm not sure how to handle queries having different ranges. Maybe return an error if they're not the same, or maybe set the range on the outer request object and have that override any per-query ranges?

[sjoeboo]

Another possible approach for this:

Do not add this functionality in Heroic directly. Grafana 7.x supports Transformations (https://grafana.com/docs/grafana/latest/panels/transformations/) do do this sort of operation (and more) directly.

Supporting such features w/ the alerting engine used internally with Heroic would be a mater of processing the transformation definitions and doing the arithmetic there, instead of implementing new interfaces in Heroic itself.

[sming]

Just a note that I picked this up as of Mon Sep 8, 2020 and may be joined by another dev - TBD.

I am writing an RFC to capture the analysis and any discussion of this feature. Currently it is only accessible to spotifiers. Once some sound proposals have been agreed upon, I will (hopefully) open it up to everyone for further discussion for a set period of time.

After a design & implementation have been agreed upon, I will begin implementation in earnest.

[sming]

Oct 8, 2020 A long-overdue update

• RFC will shortly be updated with all the plans and information I have elsewhere
• The RFC Doc
• These slides describe the RFC in a more visual fashion
• There is also heroic-benchmark which was created and used to benchmark the different libraries that support natural-language mathematic expression parsing (namely MxParser and Exp4j)