Optimize aggregation memory breaker

[jrj0823]

Signed-off-by: Renjie Jin [email protected]

Description

A new config 'search.breaker.request_level_limit' is added to prevent a single aggregation qeury from exceeding certain threshold. The agg-memory breaker is divided into two parts. First, ensuring that the memory usage of a single query does not exceed a certain threshold, and then ensuring that the total memory usage of all aggregation queries does not exceed a certain threshold.

Related Issues

Resolves #8948

Check List

• [ ] New functionality includes testing.
  • [ ] All tests pass
• [ ] New functionality has been documented.
  • [ ] New functionality has javadoc added
• [ ] Commits are signed per the DCO using --signoff
• [ ] Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[github-actions[bot]]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE :x:
• URL: https://build.ci.opensearch.org/job/gradle-check/21214/
• CommitID: 351aada2c76f63030842ee3e1885ffd804b7ba22 Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions[bot]]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE :x:
• URL: https://build.ci.opensearch.org/job/gradle-check/21238/
• CommitID: 764494b04cf8224f4d8c10d0764c841f0b93bf90 Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions[bot]]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE :x:
• URL: https://build.ci.opensearch.org/job/gradle-check/21239/
• CommitID: a6d0364342b30b5e3ffe7a9958187d10f78e4cec Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions[bot]]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE :x:
• URL: https://build.ci.opensearch.org/job/gradle-check/21243/
• CommitID: d80cd71a7da1445757b64d70fc542699aaf1c6ae Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions[bot]]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE :x:
• URL: https://build.ci.opensearch.org/job/gradle-check/21247/
• CommitID: ee9375d0f7242eb7de5a5bd550ea1ad756f7e9bc Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions[bot]]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE :x:
• URL: https://build.ci.opensearch.org/job/gradle-check/21256/
• CommitID: 424c135b15a4b955615f86b3971530148dbedc97 Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions[bot]]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE :x:
• URL: https://build.ci.opensearch.org/job/gradle-check/21362/
• CommitID: 3f223806a7fef1ce320a1a8abe3d5cfc7c1b6982 Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[opensearch-trigger-bot[bot]]

This PR is stalled because it has been open for 30 days with no activity.

[mgodwan]

@jrj0823 Could you work on addressing open comments on the PR so that the maintainers can review? I believe this should be a good enhancement to search resiliency.

[sandeshkr419]

@jrj0823 Wanted to check if you are still working on this.