OpenSearch icon indicating copy to clipboard operation
OpenSearch copied to clipboard

Avoid string out of bounds error in snapshot delete

Open andrross opened this issue 1 year ago • 12 comments

Test failure #8771 shows cases where certain random seeds trigger this case. The bug is clear: the substring() call should happen after the startsWith() check in case the blob name is shorter than the prefix length being used as the start index of the substring call. I don't yet know if/how this manifests in real deployments.

This bug is deterministically reproducible with the following seed, which also verifies the fix:

./gradlew ':server:test' --tests "org.opensearch.snapshots.SnapshotResiliencyTests.testConcurrentSnapshotDeleteAndDeleteIndex" -Dtests.seed=E816C7242330BF50

Related Issues

Resolves #8771

Check List

  • [x] New functionality includes testing.
    • [x] All tests pass
  • [x] New functionality has been documented.
    • [x] New functionality has javadoc added
  • [x] Failing checks are inspected and point to the corresponding known issue(s) (See: Troubleshooting Failing Builds)
  • [x] Commits are signed per the DCO using --signoff
  • [x] Commit changes are listed out in CHANGELOG.md file (See: Changelog)
  • [x] Public documentation issue/PR created

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. For more information on following Developer Certificate of Origin and signing off your commits, please check here.

andrross avatar Feb 15 '24 17:02 andrross

FYI @sohami @harishbhakuni @kasundra07

andrross avatar Feb 15 '24 17:02 andrross

:x: Gradle check result for 977e69ef18b828d524e708f5638bdaec4fc8bba4: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions[bot] avatar Feb 15 '24 17:02 github-actions[bot]

Compatibility status:

Checks if related components are compatible with change 09778ad

Incompatible components

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/custom-codecs.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/flow-framework.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/performance-analyzer.git]

github-actions[bot] avatar Feb 15 '24 17:02 github-actions[bot]

:grey_exclamation: Gradle check result for 20aaa8ddcfde84bca8825ab2f4a966bb551165f2: UNSTABLE

  • TEST FAILURES:
      1 org.opensearch.remotestore.RemoteIndexPrimaryRelocationIT.testPrimaryRelocationWhileIndexing

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

github-actions[bot] avatar Feb 15 '24 18:02 github-actions[bot]

Codecov Report

Attention: 25 lines in your changes are missing coverage. Please review.

Comparison is base (f73c82a) 71.35% compared to head (09778ad) 71.44%. Report is 8 commits behind head on main.

Files Patch % Lines
...ch/repositories/blobstore/BlobStoreRepository.java 3.84% 24 Missing and 1 partial :warning:
Additional details and impacted files
@@             Coverage Diff              @@
##               main   #12337      +/-   ##
============================================
+ Coverage     71.35%   71.44%   +0.09%     
+ Complexity    59818    59801      -17     
============================================
  Files          4959     4959              
  Lines        281129   281125       -4     
  Branches      40857    40854       -3     
============================================
+ Hits         200590   200855     +265     
+ Misses        63908    63602     -306     
- Partials      16631    16668      +37     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov[bot] avatar Feb 15 '24 18:02 codecov[bot]

:x: Gradle check result for d05e5fa2da83bb86de6f8aa22515b485d69fb97b: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions[bot] avatar Feb 15 '24 19:02 github-actions[bot]

:x: Gradle check result for a3e737654b98e544c0224b2554c7609e98e6a983: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions[bot] avatar Feb 15 '24 20:02 github-actions[bot]

:x: Gradle check result for 09778ad3e563fffeb6a0d76677fbbf93ec5762c1: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions[bot] avatar Feb 15 '24 21:02 github-actions[bot]

❌ Gradle check result for 09778ad: FAILURE

#9208 #9191 #10673

andrross avatar Feb 15 '24 22:02 andrross

:x: Gradle check result for 09778ad3e563fffeb6a0d76677fbbf93ec5762c1: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions[bot] avatar Feb 15 '24 23:02 github-actions[bot]

❌ Gradle check result for 09778ad: FAILURE

https://github.com/opensearch-project/OpenSearch/issues/9208 https://github.com/opensearch-project/OpenSearch/issues/9191

andrross avatar Feb 17 '24 17:02 andrross

:x: Gradle check result for 09778ad3e563fffeb6a0d76677fbbf93ec5762c1:

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions[bot] avatar Feb 17 '24 17:02 github-actions[bot]

:grey_exclamation: Gradle check result for 09778ad3e563fffeb6a0d76677fbbf93ec5762c1: UNSTABLE

  • TEST FAILURES:
      1 org.opensearch.remotestore.RemoteIndexPrimaryRelocationIT.testPrimaryRelocationWhileIndexing

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

github-actions[bot] avatar Feb 19 '24 00:02 github-actions[bot]