OpenSearch icon indicating copy to clipboard operation
OpenSearch copied to clipboard

[Feature Request] The index adds automatic force merge function to reduce segments

Open kkewwei opened this issue 1 year ago • 0 comments

Is your feature request related to a problem? Please describe

In our product, for small index (about 20g-), the frequency of writing/updating is not high, we have to frequently execute forceMerge segment to reduce the segments to improve query performance. As opensearch use TieredMergePolicy, which just merges segments of approximately equal size.

As ISM provides abality to forcemerge periodically, brings a lot of query glitches, the new created segment can't be merged quickly.

Describe the solution you'd like

Customizing/extending MergePolicy a supported API and designed for users in lucene, If we should support another MergePolicy in opensearch, which can auto merge as much as possible to reduce the number of segments.

If this is reasonable, I will follow up with how to design rules to auto force merge.

Related component

Search:Performance

Describe alternatives you've considered

No response

Additional context

No response

kkewwei avatar Oct 18 '24 06:10 kkewwei