snowflake-connector-nodejs icon indicating copy to clipboard operation
snowflake-connector-nodejs copied to clipboard

Published 20 hours ago • verified publisher icon snowflakedb

SNOW-765513: optional peer dependencies for different clouds

Open brianzinn opened this issue 2 years ago • 8 comments

  1. What did you do? I'm running snowflake-sdk in google cloud function.

  2. What did you expect to see? No messages for AWS

  3. What did you see instead? (node:1) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. Please migrate your code to use AWS SDK for JavaScript (v3)

Is there any consideration to make AWS/Azure as optional peer dependencies? This would:

  1. reduce bundle size
  2. allow the folks requesting AWS v3 to choose their version #365

brianzinn avatar Mar 21 '23 18:03 brianzinn

thank you for submitting this feature request ! we'll see how it would be feasible to implement. thank you for bearing with us !

sfc-gh-dszmolka avatar Mar 24 '23 12:03 sfc-gh-dszmolka

Just providing more data here :)

For my use case - I would like to use the snowflake-sdk parts that communicate with snowflake - and don't need to use any of the bundled in cloud SDKs.

doom-weaver avatar May 10 '23 15:05 doom-weaver

For my use case - I would like to use the snowflake-sdk parts that communicate with snowflake - and don't need to use any of the bundled in cloud SDKs.

Same. That's a perfect case for peerDependencies.

kirillgroshkov avatar May 11 '23 13:05 kirillgroshkov

If this is not fixed - I guess as a community we'll need to settle on a fork that would have these extra deps as peerDeps. We're getting close to forking it BTW.

kirillgroshkov avatar May 11 '23 13:05 kirillgroshkov

Heads up, since this issue is not fixed - I've started working on a fork for our organization, to get rid of unused dependencies that bloat our lockfiles and node_modules (and CI with npm warnings).

I've now pushed the first version of the fork and it has passed our org internal tests - all our use cases of snowflake-sdk still work (as we don't require AWS sdk, Azure, etc).

Not encouraging anyone to use it (better to fix the original package), but here's our published example: https://www.npmjs.com/package/@naturalcycles/snowflake-sdk

yarn.lock diff looks like this:

image

kirillgroshkov avatar Dec 26 '23 19:12 kirillgroshkov

thanks @kirillgroshkov - the Node.js support from snowflake appears the least supported (https://developers.snowflake.com/drivers-and-libraries/). I wrote also years ago a Node.js snowpipe SDK (https://github.com/brianzinn/snowflake-ingest-node). I haven't been following the updates lately, but it looks like they are encouraging streams now and I don't see any snowpipe support for any language anymore - looks like they are using Spark/snowpark instead of an API, so will be even less Node.js compatibility!? Probably makes sense from a data team/processing perspective, but not necessarily as part of a company ecosystem in my case even though I am proficient in Python and Java. Then again... I am not following their developer strategy!!

brianzinn avatar Dec 26 '23 20:12 brianzinn

Today I stepped one level deeper and found more unused/unnecessary dependencies to remove from package.json (or move to devDependencies):

image

kirillgroshkov avatar Dec 27 '23 17:12 kirillgroshkov

Just providing more data here :)

For my use case - I would like to use the snowflake-sdk parts that communicate with snowflake - and don't need to use any of the bundled in cloud SDKs.

Ditto

larryboymi avatar Jul 11 '24 11:07 larryboymi