datawave icon indicating copy to clipboard operation
datawave copied to clipboard

Make Ivarator Seek Threshold Configurable

Open apmoriarty opened this issue 3 years ago • 1 comments

https://github.com/NationalSecurityAgency/datawave/pull/1142 updated Ivarators to take advantage of a document range when possible. Part of this change introduced the concept of seeking through a field to the specific document id when a threshold was reached.

The default value was set to 40 consecutive hits, but this threshold should be configurable.

apmoriarty avatar Jun 25 '21 14:06 apmoriarty

Setter/getter on ShardQueryLogic which pass through the setter/getter on the config (ShardQueryConfiguration) Add the member to the ShardQueryConfiguration object with setter/getter In the DefaultQueryLogic where the QueryIterator is configured, pass the value through as an iterator option (via the IteratorSetting) In the QueryIterator, pull the value out of the options on the seek method, and pass it through to the IteratorBuildingVisitor In the IteratorBuildingVisitor pass it through to the DatawaveFieldIndexCachingiteratorJexl.

ivakegg avatar Jan 11 '22 16:01 ivakegg