snowflake-kafka-connector
snowflake-kafka-connector copied to clipboard
Document prerequisites to enable exactly-once delivery guarantees
I'm trying to run integration-tests on this project (release 1.8) to validate some changes I've added to a fork. It seems I am unable to run any test which tries to create a "client-sequencer" from the new snowflake-ingest-sdk (this code-path seems to be generally reached when one configures the connector for exactly-once delivery guarantees). The ingest-sdk reports the following unauthorization error to me:
Message: Not authorized to manage the specified object. Snowpipe APIs used to configure client, get status and insertFiles with clientInfo are disabled,
Can you tell me what I need to do to my snowflake user to enable these API endpoints so I can run all the integration tests? Thanks!
Looks like it's because the EOS feature is still in PrPr, @sfc-gh-japatel any idea about a workaround here?
There is no workaround here apart from enabling the new APIs on server side for their account. As of today:
- We are not planning to get this feature out of PrPr.
- We will discontinue the use of this config. (1.7.x wont be recommended to use), hence removal of this tests.
- We are in PuPr for our Snowpipe Streaming APIs which gives all benefits of EXACTLY_ONCE feature we announced with snowpipe last year.
Closing this issue. @chrisVia feel free to reopen again.