OpenSearch icon indicating copy to clipboard operation
OpenSearch copied to clipboard

Feature/QuerySandbox(ResourceLimitGroup) Add sandbox schema and tracking framework

Open kaushalmahi12 opened this issue 10 months ago • 4 comments

Description

As part of limiting the resource usage to the group of queries(Query Sandboxing/resourceLimitGroups). The feature is broken down into 3 independent parts.

  • CRUD APIs (Will go into the Plugin)
  • Request Tagging/Classification (Will go into the Plugin)
  • Tracking and Monitoring

This change sets up the tracking and monitoring framework which consists of

  • Schema for defining these groups/sandboxes
  • Service which will run on a 1 second interval. This class is not functional as it needs to be added in the Node.java
  • Service level settings
  • Logic to propagate the sandbox/resourceLimitGroup from coordinator to data nodes.

Remaining Changes

  • Will add the Tests for Service once the CRUD APIs are raised and merged
  • ResourceTracking tests once CRUD APIs are raised and merged
  • Enable the service

Related Issues

Resolves #[Issue number to be closed when this PR is merged]

Check List

  • [ ] New functionality includes testing.
    • [X] All tests pass
  • [ ] New functionality has been documented.
    • [X] New functionality has javadoc added
  • [ ] Failing checks are inspected and point to the corresponding known issue(s) (See: Troubleshooting Failing Builds)
  • [X] Commits are signed per the DCO using --signoff
  • [ ] Commit changes are listed out in CHANGELOG.md file (See: Changelog)
  • [ ] Public documentation issue/PR created

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. For more information on following Developer Certificate of Origin and signing off your commits, please check here.

kaushalmahi12 avatar Apr 19 '24 17:04 kaushalmahi12

:white_check_mark: Gradle check result for 5ba753570a2c039fd749fcfe799b920ee8500ff0: SUCCESS

github-actions[bot] avatar Apr 19 '24 18:04 github-actions[bot]

Codecov Report

Attention: Patch coverage is 50.50167% with 148 lines in your changes missing coverage. Please review.

Project coverage is 71.28%. Comparing base (b15cb0c) to head (5ba7535). Report is 373 commits behind head on main.

:exclamation: Current head 5ba7535 differs from pull request most recent head 3afd9db

Please upload reports for the commit 3afd9db to get more accurate results.

Files Patch % Lines
...esourceLimitsGroupResourceUsageTrackerService.java 0.00% 41 Missing :warning:
...h/cluster/metadata/ResourceLimitGroupMetadata.java 38.29% 29 Missing :warning:
...esource_limit_group/ResourceLimitGroupService.java 0.00% 23 Missing :warning:
...java/org/opensearch/cluster/metadata/Metadata.java 0.00% 18 Missing :warning:
...limit_group/ResourceLimitGroupServiceSettings.java 62.79% 14 Missing and 2 partials :warning:
...pensearch/cluster/metadata/ResourceLimitGroup.java 92.39% 2 Missing and 5 partials :warning:
...e_limit_group/module/ResourceLimitGroupModule.java 0.00% 5 Missing :warning:
...va/org/opensearch/action/search/SearchRequest.java 55.55% 2 Missing and 2 partials :warning:
...pensearch/rest/action/search/RestSearchAction.java 0.00% 1 Missing and 1 partial :warning:
...opensearch/search/internal/ShardSearchRequest.java 75.00% 0 Missing and 2 partials :warning:
... and 1 more
Additional details and impacted files
@@             Coverage Diff              @@
##               main   #13311      +/-   ##
============================================
- Coverage     71.42%   71.28%   -0.14%     
- Complexity    59978    60609     +631     
============================================
  Files          4985     5045      +60     
  Lines        282275   285730    +3455     
  Branches      40946    41358     +412     
============================================
+ Hits         201603   203685    +2082     
- Misses        63999    65152    +1153     
- Partials      16673    16893     +220     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov[bot] avatar Apr 19 '24 18:04 codecov[bot]

:x: Gradle check result for d2a19ee6898d14769534662ee1d1340ae703c8fc: null

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions[bot] avatar Apr 22 '24 21:04 github-actions[bot]

:x: Gradle check result for 3afd9db86ea3be0dc8d92d0ac397deef2d0755f2: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions[bot] avatar May 08 '24 18:05 github-actions[bot]

This PR is stalled because it has been open for 30 days with no activity.

This PR is stalled because it has been open for 30 days with no activity.

I guess we can close this PR. @kaushalmahi12 - Please reopen if needed

jainankitk avatar Sep 18 '24 23:09 jainankitk