Michael Froh

> Hi @msfroh > > ':distribution:bwc:staged:buildBwcLinuxTar' is failing and I am unable to point out what could be going wrong > > any direction would be extremely useful > >...

1. We need to decide on the possible API -- proposing something is probably a good first issue, but we would need to get consensus. 2. @lukas-vlcek called out that...

Yeah -- while (perhaps) confusing, it's longstanding behavior that search analyzers will take precedence over the index-time analyzers. Maybe we could define a new field-level analyzer parameter (`field_analyzer`?) that implicitly...

> do you have data to proof that 10 mins is the right/best timeout value? Absolutely not! It's another totally arbitrary limit that should be debated, ideally based on data....

I had an absolutely terrible idea for an alternative solution: We can keep track of how frequently a given index receives search requests while idle. If the index receives more...

I was just talking with @ruai0511 and we came up with another possible option. What if shard idle didn't *stop* refreshes altogether, but rather just made them more sparse. Right...

The discussion I had with @ruai0511 was mostly about "What is search idle really trying to solve?" Essentially, if you have a "write-only" logging use-case (or an overnight rebuild), the...

>> Flush segments every second, producing lots of small segments which need to be merged. > > We have integrated Lucene's merge-on-refresh policy a while back, that should help with...

> With the exponentially decaying refresh rate, I assume we'd still force a refresh if a search request hit the shard if it was in the increased refresh rate state...

IMO, if someone wants a staleness guarantee, they should either explicitly set the refresh interval (disabling the shard idle behavior) or issue an explicit refresh before they search.