A high-level service for pagination through the whole index

[mitar]

Which version of Elastic are you using?

[x] elastic.v7 (for Elasticsearch 7.x) [ ] elastic.v6 (for Elasticsearch 6.x) [ ] elastic.v5 (for Elasticsearch 5.x) [ ] elastic.v3 (for Elasticsearch 2.x) [ ] elastic.v2 (for Elasticsearch 1.x)

Please describe the expected behavior

There is a ScrollService which allows one to scroll through the whole index, as described here. But since the introduction of point in time (PIT) support in ElasticSearch, scrolling is not recommended anymore and instead use of PITs is. It would be great if this library would provide a high-level service which would be easy to use to scroll through the whole index. Similar to how there is a bulk processor, maybe a index processor or something, which would then internally use PITs to do all the work and retrying and so on to make sure you get all documents from the index, without any skipped or duplicated.

[olivere]

I have to look into this. What I've seen time and time again though is that you start simple, then users want to tweak "just that one" Search and Scroll and PIT setting. After a few iterations you end up with a complex beast that a) has to be maintaned, b) is hard to use, and c) would've better been added in a few lines of your own codebase anyway (so you can tweak it as desired).

Long story short: I understand the requirement. But I've seen it's often unclear where to draw the line.

[mitar]

Maybe make it equivalent to ScrollService feature wise? :-)