snowflake-ingest-java icon indicating copy to clipboard operation
snowflake-ingest-java copied to clipboard

Code refactor for Iceberg support

Open sfc-gh-alhuang opened this issue 7 months ago • 0 comments

The Iceberg support PR on feature branch includes some refactor. Merge the refactor code to reduce conflict when merging feature branch in the future.

This PR includes following changes

  1. Split StreamingIngestStage into StorageManager and StreamingIngestStorage. The StorageManager is responsible for managing storage. This is useful for streaming to Iceberg which a StorageManager could manage multiple storages.
  2. Add SnowflakeServiceClient to handle all api calls to server. As we are going to add some endpoints for Iceberg table streaming, having an all-in-one place api caller makes development easier.

sfc-gh-alhuang avatar Jul 11 '24 22:07 sfc-gh-alhuang