Support wildcard/regex for indices param in _remotestore/_restore

[linuxpi]

Description

• _remotestore/_restore will now support wildcards/regex for indices param.
• Specifying body - { "indices": ["my-index-*"] } will match all indices whose name start with my-index- and trigger remote store restore.

Related Issues

Resolves https://github.com/opensearch-project/OpenSearch/issues/8480

Check List

• [x] New functionality includes testing.
  • [x] All tests pass
• [x] New functionality has been documented.
  • [x] New functionality has javadoc added
• [x] Commits are signed per the DCO using --signoff
• [x] Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[github-actions[bot]]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE :x:
• URL: https://build.ci.opensearch.org/job/gradle-check/21086/
• CommitID: 85c230d62b3068ff68ca33ae68f500f39402556d Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions[bot]]

Gradle Check (Jenkins) Run Completed with:

• RESULT: UNSTABLE :grey_exclamation:
• TEST FAILURES:

      1 org.opensearch.action.admin.cluster.node.tasks.ResourceAwareTasksTests.testTaskResourceTrackingDuringTaskCancellation

• URL: https://build.ci.opensearch.org/job/gradle-check/21121/
• CommitID: 61aa2343d6ab4a6069d9ee34580d62c7c73bb734 Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

[codecov[bot]]

Codecov Report

Attention: Patch coverage is 77.19298% with 13 lines in your changes are missing coverage. Please review.

Project coverage is 71.66%. Comparing base (b15cb0c) to head (488ab9c). Report is 259 commits behind head on main.

Files	Patch %	Lines
...in/java/org/opensearch/common/util/IndexUtils.java	84.61%	4 Missing and 4 partials :warning:
...arch/index/recovery/RemoteStoreRestoreService.java	0.00%	4 Missing :warning:
...ava/org/opensearch/snapshots/SnapshotsService.java	0.00%	1 Missing :warning:

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #8922      +/-   ##
============================================
+ Coverage     71.42%   71.66%   +0.24%     
- Complexity    59978    61106    +1128     
============================================
  Files          4985     5052      +67     
  Lines        282275   287122    +4847     
  Branches      40946    41602     +656     
============================================
+ Hits         201603   205771    +4168     
- Misses        63999    64322     +323     
- Partials      16673    17029     +356     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[github-actions[bot]]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE :x:
• URL: https://build.ci.opensearch.org/job/gradle-check/21554/
• CommitID: 0a5e37ab66fb4ac093489ac42b070eebfe7e06c0 Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[opensearch-trigger-bot[bot]]

This PR is stalled because it has been open for 30 days with no activity. Remove stalled label or comment or this will be closed in 7 days.

[github-actions[bot]]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE :x:
• URL: https://build.ci.opensearch.org/job/gradle-check/24759/
• CommitID: 92f112f673334ada1c9bae74bb55dad2b19b2d82 Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions[bot]]

Compatibility status:

Checks if related components are compatible with change efd8d7c

Incompatible components

Incompatible components: [https://github.com/opensearch-project/security.git]

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/custom-codecs.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/flow-framework.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/sql.git]

[github-actions[bot]]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE :x:
• URL: https://build.ci.opensearch.org/job/gradle-check/24830/
• CommitID: 1866d694815564461f4e377ade4b0851ae88831d Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions[bot]]

Compatibility status:

Checks if related components are compatible with change 1866d69

Incompatible components

Incompatible components: [https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/neural-search.git]

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git]

[github-actions[bot]]

Compatibility status:

Checks if related components are compatible with change 60940e2

Incompatible components

Incompatible components: [https://github.com/opensearch-project/cross-cluster-replication.git]

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/reporting.git]

[github-actions[bot]]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE :x:
• URL: https://build.ci.opensearch.org/job/gradle-check/24846/
• CommitID: 60940e22598ec48cc3b9d318b4ae3114358c56b8 Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions[bot]]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE :x:
• URL: https://build.ci.opensearch.org/job/gradle-check/25013/
• CommitID: 8f413f52fc92bff526210ecf1bd5d3797d071500 Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions[bot]]

Compatibility status:

Checks if related components are compatible with change 8f413f5

Incompatible components

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git]

[linuxpi]

Test Failures

org.opensearch.index.codec.MultiCodecReindexIT.testReindexingMultipleCodecs org.opensearch.index.codec.MultiCodecReindexIT.classMethod org.opensearch.backwards.IndexingIT.testIndexVersionPropagation org.opensearch.backwards.IndexingIT.testSeqNoCheckpoints org.opensearch.backwards.IndexingIT.testIndexingWithReplicaOnBwcNodes org.opensearch.backwards.MixedClusterClientYamlTestSuiteIT.test {p0=indices.stats/20_translog/Translog stats on closed indices} org.opensearch.backwards.MixedClusterClientYamlTestSuiteIT.test {p0=indices.stats/13_fields/Fields - pattern} org.opensearch.backwards.MixedClusterClientYamlTestSuiteIT.test {p0=indices.stats/13_fields/Completion - pattern} org.opensearch.backwards.MixedClusterClientYamlTestSuiteIT.test {p0=indices.stats/13_fields/Completion fields - multi} org.opensearch.backwards.MixedClusterClientYamlTestSuiteIT.test {p0=cat.nodes/10_basic/Additional disk information} org.opensearch.backwards.MixedClusterClientYamlTestSuiteIT.test {p0=indices.stats/13_fields/Fields - star} org.opensearch.backwards.MixedClusterClientYamlTestSuiteIT.test {p0=indices.stats/13_fields/Completion fields - star} org.opensearch.backwards.MixedClusterClientYamlTestSuiteIT.test {p0=nodes.stats/10_basic/Nodes stats level} org.opensearch.backwards.MixedClusterClientYamlTestSuiteIT.test {p0=indices.stats/13_fields/Fielddata fields - pattern} org.opensearch.backwards.MixedClusterClientYamlTestSuiteIT.test {p0=nodes.stats/11_indices_metrics/Metric - indices _all} org.opensearch.backwards.MixedClusterClientYamlTestSuiteIT.test {p0=indices.stats/13_fields/Fields - multi} org.opensearch.backwards.MixedClusterClientYamlTestSuiteIT.test {p0=indices.stats/13_fields/Completion - all metric} org.opensearch.backwards.MixedClusterClientYamlTestSuiteIT.test {p0=indices.stats/14_groups/Groups - blank} org.opensearch.backwards.MixedClusterClientYamlTestSuiteIT.test {p0=indices.stats/12_level/Level - blank} org.opensearch.backwards.MixedClusterClientYamlTestSuiteIT.test {p0=indices.stats/13_fields/Fielddata fields - one} org.opensearch.backwards.MixedClusterClientYamlTestSuiteIT.test {p0=cat.indices/20_hidden/Test cat indices output with a hidden index with a hidden alias}

[github-actions[bot]]

Gradle Check (Jenkins) Run Completed with:

• RESULT: :x:
• URL: https://build.ci.opensearch.org/job/gradle-check/25040/
• CommitID: 2b2fc1e184a9d4ffdc2330fca940259ffc2da25c Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions[bot]]

Compatibility status:

Checks if related components are compatible with change 2b2fc1e

Incompatible components

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/reporting.git]

[linuxpi]

Test Result (1 failure / +1)

[org.opensearch.search.profile.query.QueryProfilerIT.testProfileMatchesRegular {p0={"search.concurrent_segment_search.enabled":"true"} p1=true}](https://build.ci.opensearch.org/job/gradle-check/25040/testReport/junit/org.opensearch.search.profile.query/QueryProfilerIT/testProfileMatchesRegular__p0___search_concurrent_segment_search_enabled___true___p1_true_/)

this is flaky https://github.com/opensearch-project/OpenSearch/issues/9869

Did Local retries on source branch. 2 failures in 110 tests. But different tests than what failed here

./gradlew ':server:internalClusterTest' --tests "org.opensearch.search.profile.query.QueryProfilerIT" -Dtests.iters=5

Tests with failures:
 - org.opensearch.search.profile.query.QueryProfilerIT.testProfileQuery {p0={"search.concurrent_segment_search.enabled":"true"} p1=true seed=[F3EFCB81911DEB22:39EA9A865650C0D]}
 - org.opensearch.search.profile.query.QueryProfilerIT.testProfileQuery {p0={"search.concurrent_segment_search.enabled":"true"} p1=true seed=[F3EFCB81911DEB22:4F5F2C05C46DCCB6]}

110 tests completed, 2 failed

[github-actions[bot]]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE :x:
• URL: https://build.ci.opensearch.org/job/gradle-check/25369/
• CommitID: 2b2fc1e184a9d4ffdc2330fca940259ffc2da25c Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions[bot]]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE :x:
• URL: https://build.ci.opensearch.org/job/gradle-check/25484/
• CommitID: 87eafdc724ef447920f78feb67fb967bd41856f2 Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions[bot]]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE :x:
• URL: https://build.ci.opensearch.org/job/gradle-check/25994/
• CommitID: d98d18a33cd3cf037aee84a024533200100850b3 Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions[bot]]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE :x:
• URL: https://build.ci.opensearch.org/job/gradle-check/26018/
• CommitID: 1c7b495d319e34f47ba6822902145a213e3ccb0c Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[opensearch-trigger-bot[bot]]

This PR is stalled because it has been open for 30 days with no activity.

[linuxpi]

@peternied sure. planning to spend some time on this next week, has been open for long!

[github-actions[bot]]

:x: Gradle check result for 088bedae8675a5abc3f8241daffb5e871b6887b7: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[linuxpi]

[Test Result](https://build.ci.opensearch.org/job/gradle-check/32484/testReport/) (3 failures / +2)

    [org.opensearch.http.SearchRestCancellationIT.testAutomaticCancellationMultiSearchDuringFetchPhase](https://build.ci.opensearch.org/job/gradle-check/32484/testReport/junit/org.opensearch.http/SearchRestCancellationIT/testAutomaticCancellationMultiSearchDuringFetchPhase/)
    [org.opensearch.cluster.MinimumClusterManagerNodesIT.testThreeNodesNoClusterManagerBlock](https://build.ci.opensearch.org/job/gradle-check/32484/testReport/junit/org.opensearch.cluster/MinimumClusterManagerNodesIT/testThreeNodesNoClusterManagerBlock/)
    [org.opensearch.remotestore.RemoteIndexPrimaryRelocationIT.testPrimaryRelocationWhileIndexing](https://build.ci.opensearch.org/job/gradle-check/32484/testReport/junit/org.opensearch.remotestore/RemoteIndexPrimaryRelocationIT/testPrimaryRelocationWhileIndexing/)

[github-actions[bot]]

:x: Gradle check result for f0139310a445e030950f397042e51d5ecc663ecd: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[linuxpi]

[Test Result](https://build.ci.opensearch.org/job/gradle-check/32486/testReport/) (1 failure / +1)

    [org.opensearch.cluster.decommission.DecommissionControllerTests.testTimesOut](https://build.ci.opensearch.org/job/gradle-check/32486/testReport/junit/org.opensearch.cluster.decommission/DecommissionControllerTests/testTimesOut/)

test is flaky - https://github.com/opensearch-project/OpenSearch/issues/11770

[github-actions[bot]]

:white_check_mark: Gradle check result for 14c39f28458af5bafb8b6142a8af41bcf87d0d06: SUCCESS

[peternied]

@linuxpi Thanks for the updates, could you address the failing checks? @BhumikaSaini-Amazon Could you take another look since the updates have come in?

[github-actions[bot]]

:white_check_mark: Gradle check result for 905d61651a1a033d7f17d25cb3c327e1b0c2516c: SUCCESS