scotty-window-processor icon indicating copy to clipboard operation
scotty-window-processor copied to clipboard

Published 20 hours ago • verified publisher icon TU-Berlin-DIMA

LazyEvaluation of the "Lift" function.

Open tuterbatuhan opened this issue 5 years ago • 3 comments

Hey, "Lift" function is lazily evaluated by calling the getAggValues() function, as it can be seen here: https://github.com/TU-Berlin-DIMA/scotty-window-processor/blob/a18a7dc101f814b41a416aca9785a5848221667d/flink-connector/src/main/java/de/tub/dima/scotty/flinkconnector/demo/FlinkQuantileDemo.java#L28-L32

You may consider triggering the "Lift" function when a window ends, instead of adding another operator for lazy evaluation.

tuterbatuhan avatar Sep 09 '19 15:09 tuterbatuhan

Hi @tuterbatuhan,

we will consider this for future versions of Scotty.

Best, Philipp

PhilippGrulich avatar Sep 09 '19 15:09 PhilippGrulich

Hi all, I wonder why this getAggValues() is lazily evaluated? Due to some performance concern?

jiayuasu avatar Mar 12 '22 09:03 jiayuasu

Hi @jiayuasu, The lazy evaluation, in this case, is related to the quantile computation. We do not expect a great performance benefit from changing it. If you have further questions, please feel free to contact us!

Best, Juliane

julianev avatar Mar 28 '22 10:03 julianev