druid icon indicating copy to clipboard operation
druid copied to clipboard

add google as external storage for msq export

Open pjain1 opened this issue 1 year ago • 2 comments

Description

Support for exporting msq results to gcs bucket. This is essentially copying the logic of s3 export for gs, originally done by @adarshsanjeev in this PR - https://github.com/apache/druid/pull/15689

Release note

Support for exporting MSQ results to a GCS path.


Key changed/added classes in this PR
  • GoogleExportStorageProvider.java
  • GoogleStorageConnectorModule.java

This PR has:

  • [x] been self-reviewed.
    • [ ] using the concurrency checklist (Remove this item if the PR doesn't have any relation to concurrency.)
  • [x] added documentation for new or modified features or behaviors.
  • [x] a release note entry in the PR description.
  • [x] added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
  • [ ] added or updated version, license, or notice information in licenses.yaml
  • [ ] added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
  • [x] added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
  • [ ] added integration tests.
  • [x] been tested in a test Druid cluster.

pjain1 avatar Mar 06 '24 07:03 pjain1

Thanks for the PR! I'll take a look at it shortly.

adarshsanjeev avatar Mar 06 '24 08:03 adarshsanjeev

Some minor copyedit nits. The only absolutely must fix one is the incorrect cloud provider being listed in an intro sentence

317brian avatar Mar 13 '24 19:03 317brian

thanks @317brian added your commits

pjain1 avatar Mar 28 '24 16:03 pjain1

@adarshsanjeev anything else ?

pjain1 avatar Mar 28 '24 16:03 pjain1

@adarshsanjeev yes tested on a cluster

pjain1 avatar Apr 04 '24 16:04 pjain1

@adarshsanjeev - If its looks good to you, can we merge this?

abhishekagarwal87 avatar Apr 05 '24 06:04 abhishekagarwal87

Yes, the changes look good to me! Thanks for the PR!

adarshsanjeev avatar Apr 05 '24 06:04 adarshsanjeev