snowflake-connector-net icon indicating copy to clipboard operation
snowflake-connector-net copied to clipboard

Remove cloud storage SDKs and use the REST API

Open SimbaGithub opened this issue 2 years ago • 5 comments

Based off SF# 00396900

Remove the cloud storage SDKs of AWS, Azure, and GCS and use their REST API for upload/download

SimbaGithub avatar Oct 27 '22 16:10 SimbaGithub

Codecov Report

Merging #543 (3a26a35) into master (2921db3) will not change coverage. The diff coverage is n/a.

:exclamation: Current head 3a26a35 differs from pull request most recent head 8fdca7e. Consider uploading reports for the commit 8fdca7e to get more accurate results

@@      Coverage Diff      @@
##   master   #543   +/-   ##
=============================
=============================

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

codecov-commenter avatar Nov 04 '22 03:11 codecov-commenter

CLA Assistant Lite bot All contributors have signed the CLA ✍️ ✅

github-actions[bot] avatar Nov 14 '22 17:11 github-actions[bot]

I have read the CLA Document and I hereby sign the CLA

sfc-gh-ext-simba-lf avatar Nov 14 '22 18:11 sfc-gh-ext-simba-lf

Are we going to do same change in other driver? Why we are moving away from SDK and use REST API?

This was requested as some of the customer projects also referenced the cloud SDKs which caused incompatibilities in their projects.

Another reason was due to being required to install the entire SDK when only upload/download was being used so it was requested to the REST API instead.

The python connector also uses the REST API

SimbaGithub avatar Nov 14 '22 18:11 SimbaGithub

Please benchmark before and after change.

I will work on benchmarking and update again with the results

SimbaGithub avatar Nov 14 '22 18:11 SimbaGithub