arquero icon indicating copy to clipboard operation
arquero copied to clipboard

Support nested window operations.

Open mationai opened this issue 3 years ago • 2 comments

How do I combine op.lag in eg. table.derive({ moving_average: aq.rolling(d => op.average(d.value), [-13, 0]) }) to not include current row's value?

mationai avatar Jul 08 '21 22:07 mationai

Right now, aggregate and window functions can not accept other window functions as part of an input column expression. I’d like to fix this, but I’m currently away on vacation.

In the meantime, you can simply use multiple derive calls. For example, one for the lag followed by one for the rolling average.

jheer avatar Jul 10 '21 07:07 jheer

After some more research, I've decided to shelve this enhancement request. First, not all combinations of window operators / windowed aggregates permit nesting anyway. Second, some require full materialization of the nested call results prior to evaluating the outer operator call, in which case there is no real performance benefit against simply making sequential derive calls. These observations, along with the significant development and testing time adding this feature would take, dissuades me from prioritizing it at the moment.

That said, if anyone else wants to develop a solution I would be happy to help to advise that work!

jheer avatar Oct 22 '21 22:10 jheer