mpenet
Implement Search After based on refactored scroll-chan
We noticed some performance issue because we're solely using ES scroll to retrieve pages of results of our queries for "real-time" requests, however it's strongly discouraged in the doc. Here we opt to implement Search After which is very close to scroll-chan, the difference is the necessity to have a proper ordering through :sort
Note that this PR refactored and simplified scroll-chan plus we fixed a possible bug: we always pick the last scroll, acquired from the last page, which wasn't the case before. As described in the doc, we should always use the lastest scroll.
Other note, We noticed that chan->seq doesn't properly close the chan when the seq isn't entirely consumed. We also simplified it using simple clojure functions. This issue was already here and no idea how to fix it yet :/ Any idea welcomed.
@mpenet I've revised and made new tests. Can you take a look for merging this? thank you
Hi, I somehow didn't get any notification, I will have a look at it once I have a bit of free time, probably this week-end.
@mpenet FYI we've been running with this patch in prod since a few days & it's working fine.
