peerdb icon indicating copy to clipboard operation
peerdb copied to clipboard

feat: integrate iceberg initial load support via jvm grpc

Open iamKunalGupta opened this issue 1 year ago • 1 comments

  • add quarkus with grpc for jvm
  • expose required routes for appends, changes and setup tables

Currently Gradle is used for build and dev and additionally does the following (via Quarkus Tasks):

  • Generates the corresponding java files for proto
  • Generates the GrpcService Interfaces for extending and injecting

TODOS:

  • [ ] Iceberg partitioning support
  • [x] UI Changes
  • [ ] ~Table namespace support~ planned for later
  • [x] More support
    • [x] Catalogs
      • [x] JDBC
      • [x] Hive
    • [x] FileIO
      • [x] S3
      • [x] GCS - supported via S3 + HMAC based access

iamKunalGupta avatar Jun 05 '24 20:06 iamKunalGupta

PR description should outline why gradle files need to be brought in

serprex avatar Jun 11 '24 05:06 serprex