Do not fail replica shard due to primary closure

[andrross]

Description

This commit prevents a replica shard from being failed in the case that a replication action to a replica is canceled due to the primary shard being closed.

Issues Resolved

closes #803

Check List

• [x] New functionality includes testing.
  • [x] All tests pass
• [x] New functionality has been documented.
  • [x] New functionality has javadoc added
• [x] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[github-actions[bot]]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE :x:
• URL: https://build.ci.opensearch.org/job/gradle-check/1460/
• CommitID: 0a3a31b3510b5eb89057e91447063c78486ca899

[github-actions[bot]]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE :x:
• URL: https://build.ci.opensearch.org/job/gradle-check/1956/
• CommitID: d23734935c207ba40a8b2de03363bb9a8bd25507

[github-actions[bot]]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE :x:
• URL: https://build.ci.opensearch.org/job/gradle-check/1957/
• CommitID: fef204f21786118281b0f7f29e4ec2e4257e1799

[github-actions[bot]]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE :x:
• URL: https://build.ci.opensearch.org/job/gradle-check/1958/
• CommitID: 839938daff9c7c484b9fae18bb1d94d62d1e4316

[github-actions[bot]]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE :x:
• URL: https://build.ci.opensearch.org/job/gradle-check/1959/
• CommitID: 191852e39702e3fa51d5d5285a43d0ed93327238

[github-actions[bot]]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE :x:
• URL: https://build.ci.opensearch.org/job/gradle-check/1960/
• CommitID: 1d3b8069d52649d2bf2c4f5b5c5d405593b3552a

[github-actions[bot]]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE :x:
• URL: https://build.ci.opensearch.org/job/gradle-check/1961/
• CommitID: 306b2bd2aade0865fb90662fafe2f29218310cbf

[github-actions[bot]]

Gradle Check (Jenkins) Run Completed with:

• RESULT: UNSTABLE :x:
• URL: https://build.ci.opensearch.org/job/gradle-check/1962/
• CommitID: 306b2bd2aade0865fb90662fafe2f29218310cbf

[github-actions[bot]]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE :x:
• URL: https://build.ci.opensearch.org/job/gradle-check/2095/
• CommitID: b27b38e4cca5fb73f451a16fadcf37852836b42d

[andrross]

@itiyama @Bukhtawar I've added a test in CorruptedFileIT that deterministically fails without this fix and passes with it.

[github-actions[bot]]

Gradle Check (Jenkins) Run Completed with:

• RESULT: SUCCESS :white_check_mark:
• URL: https://build.ci.opensearch.org/job/gradle-check/2097/
• CommitID: 5244a32684ef58c0033c72dc50922200fa06f57d

[codecov-commenter]

Codecov Report

Merging #4133 (692e140) into main (5dd7947) will decrease coverage by 0.09%. The diff coverage is 83.33%.

@@             Coverage Diff              @@
##               main    #4133      +/-   ##
============================================
- Coverage     70.78%   70.69%   -0.10%     
+ Complexity    57309    57217      -92     
============================================
  Files          4610     4611       +1     
  Lines        275105   275108       +3     
  Branches      40283    40284       +1     
============================================
- Hits         194745   194489     -256     
- Misses        64093    64343     +250     
- Partials      16267    16276       +9     

Impacted Files	Coverage Δ
...earch/index/shard/PrimaryShardClosedException.java	50.00% <50.00%> (ø)
.../main/java/org/opensearch/OpenSearchException.java	93.04% <100.00%> (+1.44%)	:arrow_up:
...support/replication/PendingReplicationActions.java	92.85% <100.00%> (-0.48%)	:arrow_down:
...tion/support/replication/TransportWriteAction.java	81.34% <100.00%> (+0.89%)	:arrow_up:
...adonly/AddIndexBlockClusterStateUpdateRequest.java	0.00% <0.00%> (-75.00%)	:arrow_down:
...readonly/TransportVerifyShardIndexBlockAction.java	9.75% <0.00%> (-58.54%)	:arrow_down:
...a/org/opensearch/client/cluster/ProxyModeInfo.java	0.00% <0.00%> (-55.00%)	:arrow_down:
...n/admin/indices/readonly/AddIndexBlockRequest.java	17.85% <0.00%> (-53.58%)	:arrow_down:
...ch/transport/ReceiveTimeoutTransportException.java	50.00% <0.00%> (-50.00%)	:arrow_down:
...pensearch/indices/breaker/CircuitBreakerStats.java	27.77% <0.00%> (-41.67%)	:arrow_down:
... and 459 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

[github-actions[bot]]

Gradle Check (Jenkins) Run Completed with:

• RESULT: SUCCESS :white_check_mark:
• URL: https://build.ci.opensearch.org/job/gradle-check/2098/
• CommitID: 692e140cf35114149e7bd8cffca5e511551c1b8f