lucene icon indicating copy to clipboard operation
lucene copied to clipboard

LUCENE-10425:PostingsEnum supports to return current index of postings

Open wjp719 opened this issue 3 years ago • 4 comments

PostingsEnum supports to return current index of postings

As we known, the docId list in .doc is like an array of integer, every element of this array is a docId, when we call nextDoc or advance method, it will move to another element of this array. this pr support to return current index of the element in the posting list. We can use this method to find the diff value when we call advance method continuously, the diff value means we skip how many doc in this docId list.

#11461

wjp719 avatar Feb 17 '22 12:02 wjp719

I'm concerned exposing this could limit future implementations by requiring them to support it. Do you have a proposed use case that would justify needing to add this? Maybe post to [email protected] explaining the use case?

msokolov avatar Feb 17 '22 16:02 msokolov

@msokolov Hi, I describe a use case in this issue https://issues.apache.org/jira/browse/LUCENE-10425 , thanks a lot

wjp719 avatar Feb 18 '22 04:02 wjp719

The use-case makes sense to me, but like @msokolov I'm worried that this could limit future implementations. For instance if we were to encode postings as a bitset, we wouldn't be able to get the current index in constant time like the current postings can, we'd have to count all one bits.

jpountz avatar Mar 05 '22 17:03 jpountz

Thinking through this a bit more, I imagine that this is the reason why you made this API optional. We still like the set the bar quite high to add new APIs to PostingsEnum so I'd like to think a bit more about what problems this would be solving. I'll leave some thoughts on the JIRA issue.

jpountz avatar Mar 05 '22 18:03 jpountz

Thanks @wjp719.

It looks like this is a nice opto for narrow use cases and the concern is adding a new API, especially to such a hot class as PostingsEnum, needs to meet a high bar for helping many uses cases, and maybe this change doesn't? Please reopen if you feel strongly @wjp719?

mikemccand avatar Nov 02 '23 10:11 mikemccand