[Segment Replication] Update getLatestReplicationCheckpoint for closed indices

[dreamer-89]

Signed-off-by: Suraj Singh [email protected]

Description

This change updates the IndexShard.getLatestReplicationCheckpoint to return empty checkpoint for closed indices. This change is needed as when indices are closed, shard info is still shared among nodes via transport actions as mentioned in https://github.com/opensearch-project/OpenSearch/issues/3823#issuecomment-1247439612

Issues Related

https://github.com/opensearch-project/OpenSearch/issues/3823

Check List

• [ ] New functionality includes testing.
  • [ ] All tests pass
• [ ] New functionality has been documented.
  • [ ] New functionality has javadoc added
• [x] Commits are signed per the DCO using --signoff
• [x] Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[github-actions[bot]]

Gradle Check (Jenkins) Run Completed with:

• RESULT: SUCCESS :white_check_mark:
• URL: https://build.ci.opensearch.org/job/gradle-check/3128/
• CommitID: 577ff76db8f899753fdcbe6a82c2cfa1a940e9e6

[github-actions[bot]]

Gradle Check (Jenkins) Run Completed with:

• RESULT: SUCCESS :white_check_mark:
• URL: https://build.ci.opensearch.org/job/gradle-check/3129/
• CommitID: f130f366a17f6e73a26017be37cbe0416c474ab9

[github-actions[bot]]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE :x:
• URL: https://build.ci.opensearch.org/job/gradle-check/3154/
• CommitID: ee136103712ba33f1a0858e44fa8dad359383244

[github-actions[bot]]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE :x:
• URL: https://build.ci.opensearch.org/job/gradle-check/3155/
• CommitID: 9ab5caf8575d08c8f26fa0bf995aed933b014acb

[github-actions[bot]]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE :x:
• URL: https://build.ci.opensearch.org/job/gradle-check/3156/
• CommitID: 2759e47196ddec6780c417c41e59b7006b0113a1

[github-actions[bot]]

Gradle Check (Jenkins) Run Completed with:

• RESULT: SUCCESS :white_check_mark:
• URL: https://build.ci.opensearch.org/job/gradle-check/3157/
• CommitID: 703427f7a209809809c72b7687afcdfdca32cd76

[codecov-commenter]

Codecov Report

Merging #4532 (703427f) into main (72e6801) will increase coverage by 0.12%. The diff coverage is 80.00%.

@@             Coverage Diff              @@
##               main    #4532      +/-   ##
============================================
+ Coverage     70.62%   70.74%   +0.12%     
- Complexity    57263    57368     +105     
============================================
  Files          4617     4617              
  Lines        275499   275509      +10     
  Branches      40331    40332       +1     
============================================
+ Hits         194562   194909     +347     
+ Misses        64528    64259     -269     
+ Partials      16409    16341      -68     

Impacted Files	Coverage Δ
...in/java/org/opensearch/index/shard/IndexShard.java	69.69% <80.00%> (+0.32%)	:arrow_up:
...n/indices/forcemerge/ForceMergeRequestBuilder.java	0.00% <0.00%> (-75.00%)	:arrow_down:
...n/admin/cluster/node/tasks/get/GetTaskRequest.java	30.30% <0.00%> (-63.64%)	:arrow_down:
.../indices/forcemerge/TransportForceMergeAction.java	25.00% <0.00%> (-58.34%)	:arrow_down:
.../java/org/opensearch/node/NodeClosedException.java	50.00% <0.00%> (-50.00%)	:arrow_down:
...ch/transport/ReceiveTimeoutTransportException.java	50.00% <0.00%> (-50.00%)	:arrow_down:
...cluster/coordination/PendingClusterStateStats.java	20.00% <0.00%> (-48.00%)	:arrow_down:
...opensearch/persistent/PersistentTasksExecutor.java	22.22% <0.00%> (-44.45%)	:arrow_down:
...h/action/ingest/SimulateDocumentVerboseResult.java	60.71% <0.00%> (-39.29%)	:arrow_down:
.../admin/cluster/node/tasks/get/GetTaskResponse.java	28.57% <0.00%> (-35.72%)	:arrow_down:
... and 467 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

[dreamer-89]

@mch2 : Addressed the review comments, can you please have a look ?

[github-actions[bot]]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE :x:
• URL: https://build.ci.opensearch.org/job/gradle-check/3270/
• CommitID: fd9987023714c483e884b4b561a46a8696365d35

[dreamer-89]

Flaky test tracked in https://github.com/opensearch-project/OpenSearch/issues/4162

REPRODUCE WITH: ./gradlew ':server:internalClusterTest' --tests "org.opensearch.cluster.routing.allocation.decider.DiskThresholdDeciderIT.testRestoreSnapshotAllocationDoesNotExceedWatermark" -Dtests.seed=A21B2927A9499A55 -Dtests.security.manager=true -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=sr-Latn-ME -Dtests.timezone=America/Detroit -Druntime.java=17

org.opensearch.cluster.routing.allocation.decider.DiskThresholdDeciderIT > testRestoreSnapshotAllocationDoesNotExceedWatermark FAILED
    java.lang.AssertionError: 
    Expected: an empty collection
         but: <[[rjewwteugy][3], node[xWCKx2aaRSSe57cBd-X-7w], [P], s[STARTED], a[id=QQKbs0HDSBSGbt78lZOipw], [rjewwteugy][1], node[xWCKx2aaRSSe57cBd-X-7w], [P], s[STARTED], a[id=HAVu-gyGTUqfxhOBO40uhw]]>
        at __randomizedtesting.SeedInfo.seed([A21B2927A9499A55:B0A38F46C2F085ED]:0)
        at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:18)
        at org.junit.Assert.assertThat(Assert.java:964)
        at org.junit.Assert.assertThat(Assert.java:930)
        at org.opensearch.cluster.routing.allocation.decider.DiskThresholdDeciderIT.lambda$testRestoreSnapshotAllocationDoesNotExceedWatermark$2(DiskThresholdDeciderIT.java:260)
        at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1049)
        at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1022)
        at org.opensearch.cluster.routing.allocation.decider.DiskThresholdDeciderIT.testRestoreSnapshotAllocationDoesNotExceedWatermark(DiskThresholdDeciderIT.java:260)

[github-actions[bot]]

Gradle Check (Jenkins) Run Completed with:

• RESULT: SUCCESS :white_check_mark:
• URL: https://build.ci.opensearch.org/job/gradle-check/3300/
• CommitID: fd9987023714c483e884b4b561a46a8696365d35

[anasalkouz]

@dreamer-89 Is this still required? it's pending for review for a while

[dreamer-89]

@dreamer-89 Is this still required? it's pending for review for a while

@anasalkouz : Yes, this is needed and appears while running ClosedIndices* integration tests. This was initially needed for CCR (as mentioned in description) but now appears to be a valid use case related to closed indices. I will work on this PR.

[dreamer-89]

Closing this stale PR, tracking fix in https://github.com/opensearch-project/OpenSearch/issues/6828