euphoria icon indicating copy to clipboard operation
euphoria copied to clipboard

euphoria-flink: optimize sort

Open horkyada opened this issue 7 years ago • 0 comments

Sort consists of sorting elements by a composite key <Window, ExtractedObject>. SortTranslator for flink executor relies on GenericTypeComparator for Window as well as for user defined extracted object which causes a huge performance overhead because of deserializing both objects on every compare.

  • we can define our custom TypeComparator for known windows, or just emit NormalizableKey instead of the window
  • however without knowing the extracted objects' type we are not able to tune the comparing of those objects

horkyada avatar Apr 25 '17 13:04 horkyada