Move query categorization changes to query insights plugin

[deshsidd]

Query categorization changes to increment counters for search query related metrics currently resides on the search path and occurs before the request.

Move these changes to the query insights plugin and make sure the incrementing of counters happens separately from the search path.

Another option is to keep query categorization changes as is. However, this will lead to additional overhead on the search path. Furthermore, we need to tie query latency, cpu, memory with the query categorization data which will only be possible if we increment the counters after the request is completed and the query latency and resource usage data resides inside the plugin.

To support the above and to prevent doing these counter increments on the search path, we need to move query categorization changes to the query insights plugin.

Related Issues

Resolves #14527 Addresses #11596

Check List

• [x] Functionality includes testing.
• [x] API changes companion pull request created, if applicable.
• [x] Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[github-actions[bot]]

:x: Gradle check result for 9e5a09addaee2c8f85d67d9ef67c66eacdb65032: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions[bot]]

:x: Gradle check result for af58c5f2bc196d2ab4f71e135d86667d0cfcab07: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions[bot]]

:x: Gradle check result for 2bd1a05a18585bff161baa1cad1ffa59ecd3ae76: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions[bot]]

:x: Gradle check result for 68a226e49448ea595a1df4f1d8bd04a412aff998: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions[bot]]

:x: Gradle check result for 8dc38c472eaa7be18b10beb091842905684127ea: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions[bot]]

:x: Gradle check result for a4ad58e3138cf5e521cc00000342173d103523cf: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions[bot]]

:x: Gradle check result for cdce692fbf3c676965b1670b9c3d4a1e657979ea: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions[bot]]

:x: Gradle check result for c58db2d49ca2ec4fdfa13476ab35f6a258cc48ac: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions[bot]]

:x: Gradle check result for c056aaa8a0a90a9964dc24fa62a8fefc701e72a8: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions[bot]]

:x: Gradle check result for 2a6c3b1313cfd01665309ca50b6734325d8573f9: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions[bot]]

:x: Gradle check result for 8ede94322debef443aede1d4124db2570bf41b70: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions[bot]]

:x: Gradle check result for 373e1bde051a458ff6c239c309cb51e23bed49fb: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions[bot]]

:x: Gradle check result for 9836921eeddf9315f005caa4f990c3f24de0516f: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions[bot]]

:x: Gradle check result for 093a79195332d6422a385d435c7b792e511cd85b: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions[bot]]

:x: Gradle check result for 827085ea1e3d0bd53dedb88131415ea11d33df30: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions[bot]]

:x: Gradle check result for dc1fefbd0d217e934106729cf0987797c6ded920: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions[bot]]

:x: Gradle check result for 0906513b3430eb67c1ec61681d141dd1f043503a: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions[bot]]

:white_check_mark: Gradle check result for be6527b95c96fafe672b1840d266296d84b2388e: SUCCESS

[codecov[bot]]

Codecov Report

Attention: Patch coverage is 67.69231% with 21 lines in your changes missing coverage. Please review.

Project coverage is 71.78%. Comparing base (a34270d) to head (eb8ebd8). Report is 5 commits behind head on main.

Files	Patch %	Lines
...in/insights/core/service/QueryInsightsService.java	46.66%	7 Missing and 1 partial :warning:
...nsearch/plugin/insights/rules/model/Attribute.java	73.91%	3 Missing and 3 partials :warning:
...re/service/categorizer/SearchQueryCategorizer.java	37.50%	5 Missing :warning:
...categorizer/SearchQueryAggregationCategorizer.java	50.00%	1 Missing :warning:
...insights/settings/QueryCategorizationSettings.java	50.00%	1 Missing :warning:

Additional details and impacted files

@@            Coverage Diff             @@
##               main   #14528    +/-   ##
==========================================
  Coverage     71.77%   71.78%            
+ Complexity    62302    62282    -20     
==========================================
  Files          5125     5127     +2     
  Lines        292473   292547    +74     
  Branches      42258    42269    +11     
==========================================
+ Hits         209912   209991    +79     
+ Misses        65324    65213   -111     
- Partials      17237    17343   +106     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[github-actions[bot]]

:grey_exclamation: Gradle check result for eb8ebd85c7e05733e69f145c9f85228d45bc46f9: UNSTABLE

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

[github-actions[bot]]

:x: Gradle check result for d419ca5d20c50e199cb4da893471350024924198: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions[bot]]

:x: Gradle check result for d6b29c7981a28478b6360e0e1bbbe1bc74735ce0: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions[bot]]

:x: Gradle check result for 9b80e9055dda29d3aa0ca7f83fffa9ea21e03fe4: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[jainankitk]

@deshsidd - This PR can be discarded in favor of opensearch-project/query-insights#16, right?

[deshsidd]

Closing in favor of : https://github.com/opensearch-project/OpenSearch/pull/14528/files