fix: The aggs result of NestedAggregator with sub NestedAggregator may be not accurately

[kkewwei]

Signed-off-by: kkewwei [email protected]

Description

The aggs result of NestedAggregator with sub NestedAggregator may be not accurately, it depends on the face that the parentDocId > childDocId in search, but it may be wrong.

Related Issues

Resolves #13303

Check List

• [x] New functionality includes testing.
  • [x] All tests pass
• [x] New functionality has been documented.
  • [x] New functionality has javadoc added
• [x] Failing checks are inspected and point to the corresponding known issue(s) (See: Troubleshooting Failing Builds)
• [x] Commits are signed per the DCO using --signoff
• [x] Commit changes are listed out in CHANGELOG.md file (See: Changelog)
• [x] Public documentation issue/PR created

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[github-actions[bot]]

:x: Gradle check result for 1087f86c027f4b95678861c6954807ba57dfdb73: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions[bot]]

:x: Gradle check result for 849dad4e5f253694d214e2886f907bb0eb069ce1: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions[bot]]

:x: Gradle check result for b7107616e589a43014e406c0a14ca643496118a0: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions[bot]]

:x: Gradle check result for e9775d73a3687b2b63221b164c983f93f822b1b0: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions[bot]]

:x: Gradle check result for 71d50694cc879ad224026add676bad9d2d238f3d: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions[bot]]

:x: Gradle check result for f264a0b92874b12ce924b22bf94bda172c664111: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[jed326]

@kkewwei thanks for opening the PR, it looks like the existing nested agg integration tests are failing after your change, could you please take a look? https://build.ci.opensearch.org/job/gradle-check/37514/#showFailuresLink

Also would you mind adding a changelog item as well? Thanks!

[kkewwei]

@jed326 I will process soon.

[github-actions[bot]]

:x: Gradle check result for 0a0107394fd81aa229e79c8e1dcb2f648e4fbe0c: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions[bot]]

:x: Gradle check result for 9c0784dc4062eca42e204213451c75d6383ded41: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions[bot]]

:white_check_mark: Gradle check result for 494bd6734acddc59c55bf483990b98f6fae4b70b: SUCCESS

[codecov[bot]]

Codecov Report

Attention: Patch coverage is 80.00000% with 6 lines in your changes missing coverage. Please review.

Project coverage is 71.79%. Comparing base (b15cb0c) to head (d3f00cc). Report is 469 commits behind head on main.

Files	Patch %	Lines
...h/aggregations/bucket/nested/NestedAggregator.java	80.00%	1 Missing and 5 partials :warning:

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #13324      +/-   ##
============================================
+ Coverage     71.42%   71.79%   +0.37%     
- Complexity    59978    62201    +2223     
============================================
  Files          4985     5125     +140     
  Lines        282275   292342   +10067     
  Branches      40946    42242    +1296     
============================================
+ Hits         201603   209894    +8291     
- Misses        63999    65166    +1167     
- Partials      16673    17282     +609     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[github-actions[bot]]

:grey_exclamation: Gradle check result for 1a279cb6427ebe5067c17f4ea6555706144827b0: UNSTABLE

• TEST FAILURES:

      1 org.opensearch.cluster.coordination.AwarenessAttributeDecommissionIT.testConcurrentDecommissionAction
      1 org.opensearch.cluster.MinimumClusterManagerNodesIT.testThreeNodesNoClusterManagerBlock

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

[github-actions[bot]]

:grey_exclamation: Gradle check result for a95ca762fe0b796daf6ed0c7cfe4361f5c06fd3f: UNSTABLE

• TEST FAILURES:

      3 org.opensearch.cluster.MinimumClusterManagerNodesIT.testThreeNodesNoClusterManagerBlock

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

[github-actions[bot]]

:white_check_mark: Gradle check result for ef1dfb8d4ec947e94cc5ea570921377966b91e08: SUCCESS

[kkewwei]

@jed326 please help review the code.

[kkewwei]

❕ Gradle check result for a95ca76: UNSTABLE

• TEST FAILURES:

      3 org.opensearch.cluster.MinimumClusterManagerNodesIT.testThreeNodesNoClusterManagerBlock

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

testThreeNodesNoClusterManagerBlock #10006

[jed326]

Thanks @kkewwei, could you add an entry to CHANGELOG.md for this as well?

[jed326]

Other than that I think this PR looks good to me, @msfroh @reta mind giving a maintainer review here as well?

[github-actions[bot]]

:white_check_mark: Gradle check result for 78cb071bbb2342374a1ec5135a03a567da978f07: SUCCESS

[github-actions[bot]]

:x: Gradle check result for 8393609887810e44d7261bd00c614567157b9fcd: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions[bot]]

:x: Gradle check result for f3e4c6f83694042c5d7c27566ff2be1f97ef3254: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[kkewwei]

❌ Gradle check result for f3e4c6f: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

testReadRangeBlobWithRetries #6090 testIndexingWithReplicaOnBwcNodes #9407 testSeqNoCheckpoints #9407 testIndexingWithPrimaryOnBwcNodes #9407 testIndexVersionPropagation #9407 MixedClusterClientYamlTestSuiteIT #2440

[github-actions[bot]]

:x: Gradle check result for 6a222a8d95d8653ea161980b6d6dc7c439d11d60: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[kkewwei]

❌ Gradle check result for 6a222a8: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

testReadRangeBlobWithRetries https://github.com/opensearch-project/OpenSearch/issues/6090 testIndexingWithReplicaOnBwcNodes https://github.com/opensearch-project/OpenSearch/issues/9407 testSeqNoCheckpoints https://github.com/opensearch-project/OpenSearch/issues/9407 testIndexingWithPrimaryOnBwcNodes https://github.com/opensearch-project/OpenSearch/issues/9407 testIndexVersionPropagation https://github.com/opensearch-project/OpenSearch/issues/9407 MixedClusterClientYamlTestSuiteIT https://github.com/opensearch-project/OpenSearch/issues/2440

[jed326]

Thanks @kkewwei , could you try to rebase from main to see if that fixes the flaky tests? We need to get the checks to green to be able to merge.

[github-actions[bot]]

:white_check_mark: Gradle check result for e4f36eb0bec6739c4c70c594c31b516280eeca3c: SUCCESS

[kkewwei]

@jed326 it's ok now, please see again.

[dblock]

Can @msfroh or @andrross take a look please? Or volunteer another maintainer?

[jed326]

@reta @msfroh @andrross would one of you be able to give a maintainer review here?

Thanks!