unipop icon indicating copy to clipboard operation
unipop copied to clipboard

Elasticsearch Scroll API

Open rmagen opened this issue 8 years ago • 1 comments

Use elasticsearch query Scroll API to iterate many results in the StartStep. https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-scroll.html

According to the docs "Scrolling is not intended for real time user requests, but rather for processing large amounts of data, e.g. in order to reindex the contents of one index into a new index with a different configuration". Should we use Scroll in Unipop? If so, should we always use it? maybe only in StartStep (Assuming most times it will iterate a large amount of results).

The scroll functionality is already implemented in QueryIterator, but its currently unused.

rmagen avatar May 11 '16 12:05 rmagen

need to implement #103 first

seanbarzilay avatar Dec 28 '16 10:12 seanbarzilay