snowflake-ingest-java
snowflake-ingest-java copied to clipboard
Code refactor for Iceberg support
The Iceberg support PR on feature branch includes some refactor. Merge the refactor code to reduce conflict when merging feature branch in the future.
This PR includes following changes
- Split
StreamingIngestStage
intoStorageManager
andStreamingIngestStorage
. TheStorageManager
is responsible for managing storage. This is useful for streaming to Iceberg which aStorageManager
could manage multiple storages. - Add
SnowflakeServiceClient
to handle all api calls to server. As we are going to add some endpoints for Iceberg table streaming, having an all-in-one place api caller makes development easier.