OpenSearch icon indicating copy to clipboard operation
OpenSearch copied to clipboard

[GRPC] Make transport-grpc extensible to other plugins

Open karenyrx opened this issue 6 months ago • 4 comments

Description

This PR refactors the GRPC transport module to provide extension points for plugins. The goal is to make it possible for third-party and internal plugins to register custom GRPC services, enhancing the flexibility and extensibility of the transport layer.

Test plan

  1. New and old unit tests still passing per ./gradlew :plugins:transport-grpc:test
  2. Existing GRPC query types still working. Example for MatchAll:
grpcurl -import-path ~/OpenSearch -d '{"size": 10000, "request_body":{"query":{"match_all":{}}}}' -proto ~/OpenSearch/protos/services/search_service.proto -plaintext localhost:9400 org.opensearch.protobufs.services.SearchService/Search

{
  "responseBody": {
    "took": "15",
    "timedOut": false,
    "shards": {},
    "hits": {
      "total": {
        "totalHits": {
          "relation": "TOTAL_HITS_RELATION_EQ"
        }
      },
      "maxScore": {
        "floatValue": 0
      }
    }
  }
}
  1. (WIP) Implementations of this interface (e.g. from the KNN plugin) are working. Tested on a test cluster in tandem with this PR in the KNN repo

Related Issues

https://github.com/opensearch-project/OpenSearch/issues/18513

Check List

  • [ ] Functionality includes testing.
  • [ ] API changes companion pull request created, if applicable.
  • [ ] Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. For more information on following Developer Certificate of Origin and signing off your commits, please check here.

karenyrx avatar Jun 13 '25 15:06 karenyrx

:white_check_mark: Gradle check result for 968cc2fb1a30a5aeeb06ae6a88ab2a73f8149d3c: SUCCESS

github-actions[bot] avatar Jun 13 '25 17:06 github-actions[bot]

Codecov Report

:x: Patch coverage is 77.77778% with 34 lines in your changes missing coverage. Please review. :white_check_mark: Project coverage is 72.76%. Comparing base (3b7603e) to head (69e2eef). :warning: Report is 5 commits behind head on main.

Files with missing lines Patch % Lines
...quest/search/query/TermQueryBuilderProtoUtils.java 54.05% 16 Missing and 1 partial :warning:
...arch/query/QueryBuilderProtoConverterRegistry.java 82.00% 8 Missing and 1 partial :warning:
...g/opensearch/plugin/transport/grpc/GrpcPlugin.java 80.00% 2 Missing and 2 partials :warning:
...arch/query/MatchAllQueryBuilderProtoConverter.java 83.33% 0 Missing and 1 partial :warning:
...rch/query/MatchNoneQueryBuilderProtoConverter.java 83.33% 0 Missing and 1 partial :warning:
...t/search/query/TermQueryBuilderProtoConverter.java 83.33% 0 Missing and 1 partial :warning:
.../search/query/TermsQueryBuilderProtoConverter.java 83.33% 0 Missing and 1 partial :warning:
Additional details and impacted files
@@             Coverage Diff              @@
##               main   #18516      +/-   ##
============================================
+ Coverage     72.67%   72.76%   +0.08%     
- Complexity    68610    68683      +73     
============================================
  Files          5577     5582       +5     
  Lines        315375   315510     +135     
  Branches      45772    45789      +17     
============================================
+ Hits         229209   229568     +359     
+ Misses        67613    67357     -256     
- Partials      18553    18585      +32     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:rocket: New features to boost your workflow:
  • :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

codecov[bot] avatar Jun 13 '25 17:06 codecov[bot]

:x: Gradle check result for f5a22431b9a3fd9c3eb8277acf8121f842331195: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions[bot] avatar Jun 13 '25 22:06 github-actions[bot]

:white_check_mark: Gradle check result for f5a22431b9a3fd9c3eb8277acf8121f842331195: SUCCESS

github-actions[bot] avatar Jun 14 '25 00:06 github-actions[bot]

:white_check_mark: Gradle check result for 3d3b570022272e3e8fb06b9c97bd4b7f6f4899a0: SUCCESS

github-actions[bot] avatar Jul 23 '25 04:07 github-actions[bot]

:white_check_mark: Gradle check result for b15e64052f637debea842cf0b04414f20bf7ad20: SUCCESS

github-actions[bot] avatar Jul 28 '25 11:07 github-actions[bot]

:white_check_mark: Gradle check result for 380ef217362a613df1e6e1efe0796c4f65684cef: SUCCESS

github-actions[bot] avatar Jul 30 '25 00:07 github-actions[bot]

:x: Gradle check result for 57322363714bba3e5e1bb8fb09d75d2e3b7ca026: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions[bot] avatar Jul 30 '25 19:07 github-actions[bot]

:x: Gradle check result for f959e3e8e3c5fd2ba5897b406fa967096cb52c1f: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions[bot] avatar Jul 30 '25 20:07 github-actions[bot]

:grey_exclamation: Gradle check result for 69e2eef45847b4ac5492064323daed845443f03e: UNSTABLE

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

github-actions[bot] avatar Jul 30 '25 21:07 github-actions[bot]