druid icon indicating copy to clipboard operation
druid copied to clipboard

gRPC query extension

Open findingrish opened this issue 1 year ago • 0 comments

Revives https://github.com/apache/druid/pull/14024 and additionally supports,

  • Native queries
  • gRPC health check endpoint

This PR doesn't have the shaded module for packaging gRPC and Guava libraries since grpc-query module uses the same Guava version as that of Druid.

The response is gRPC-specific. It provides the result schema along with the results as a binary "blob". Results can be in CSV, JSON array lines or as an array of Protobuf objects. If using Protobuf, the corresponding class must be installed along with the rRPC query extension so it is available to the Broker at runtime.

Example usage

Sample request,

QueryRequest.newBuilder()
            .setQuery("SELECT * FROM foo")
            .setResultFormat(QueryResultFormat.CSV)
            .setQueryType(QueryOuterClass.QueryType.SQL)
            .build();

When using Protobuf response format, bundle up your Protobuf classes into a jar file, and place that jar file in the $DRUID_HOME/extensions/grpc-query directory. Specify the response Protobuf message name in the request.

QueryRequest.newBuilder()
            .setQuery("SELECT dim1, dim2, dim3, cnt, m1, m2, unique_dim1, __time AS "date" FROM foo")
            .setQueryType(QueryOuterClass.QueryType.SQL)
            .setProtobufMessageName(QueryResult.class.getName())
            .setResultFormat(QueryResultFormat.PROTOBUF_INLINE)
            .build();    
            
Response message 
message QueryResult {
  string dim1 = 1;
  string dim2 = 2;
  string dim3 = 3;
  int64 cnt = 4;
  float m1 = 5;
  double m2 = 6;
  bytes unique_dim1 = 7;
  google.protobuf.Timestamp date = 8;
}          

Release note

This is a "contrib" extension. We don't seem to document such extensions in Druid itself. The README.md can serve as documentation instead.



This PR has:

  • [X] been self-reviewed.
    • [ ] using the concurrency checklist (Remove this item if the PR doesn't have any relation to concurrency.)
  • [X] added documentation for new or modified features or behaviors.
  • [X] a release note entry in the PR description.
  • [X] added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
  • [X] added or updated version, license, or notice information in licenses.yaml
  • [X] added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
  • [X] added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
  • [ ] added integration tests.
  • [x] been tested in a test Druid cluster.

findingrish avatar Feb 27 '24 18:02 findingrish