feathub
feathub copied to clipboard
Optimize sliding window late data insertion with skip list
SlidingWindowKeyedProcessFunction.java:
if (timestamps.isEmpty() || timestamps.get(timestamps.size() - 1) < timestamp) {
timestampList.add(timestamp);
} else {
// TODO: Use skip list to optimize the performance of inserting late data.
for (int insertionIdx = 0; insertionIdx < timestamps.size(); insertionIdx++) {
if (timestamps.get(insertionIdx) < timestamp) {
continue;
}
timestamps.add(insertionIdx, timestamp);
break;
}
timestampList.update(timestamps);
}