flux icon indicating copy to clipboard operation
flux copied to clipboard

Ideas for more performant statistical calculations

Open adamperlin opened this issue 6 years ago • 0 comments

From ifql created by nathanielc : influxdata/ifql#69

Currently IFQL has methods for count, sum, mean, stddev, and skew. We will probably add another for kurtosis.

Two ideas:

  • Add a stats function that lets you simply declare which you want, then the correct underlying implementation is picked depending on the set.
  • Create an mapreduce type, where procedures can declare themselves as an mapreducable type. An mapreducable type knows how to run in parallel and merge intermediate results.

See https://www.johndcook.com/blog/skewness_kurtosis/ as an example of single type that can compute all stats at once and be performed in a map reduce computation style.

adamperlin avatar Jul 03 '18 22:07 adamperlin