dynamodb-toolbox
dynamodb-toolbox copied to clipboard
jeremydaly
Support AWS SDK for JavaScript v3
it will be nice to switch to the modularized AWS SDK for JavaScript v3 (https://github.com/aws/aws-sdk-js-v3) since it reached gamma status. AWS Amplify is already using it.
dynamodb-toolbox doesn't require the full AWS SDK and will reduce the size of the bundle.
FWIW, it's a bit premature since AWS SDK for JavaScript v3 doesn't support DocumentClient yet.
I was under the impression that v3 would have the same semantics as v2, just a different import method. If that's the case, I don't think any changes will be needed, especially since v0.2 of this library just requires the DocumentClient.
I'm just trying this out today and sadly this doesn't seem to be the case.
DocumentClient doesn't seem to be part of https://github.com/aws/aws-sdk-js-v3 anymore and the API for DynamoDB in "@aws-sdk/client-dynamodb" seems to be at least slightly different to the one DocumentClient was providing.
Yes, we discovered this right after v3 was released. Still looking for suggestions for handling this, but it would be really nice if AWS would reimplement the DocumentClient.
Would the easiest option in the meantime be to create a proxy for DocumentClient?
I'll have some spare time over the next couple of weeks to see if I can add if you think it's a reasonable option.
It’s either that, or we build the marshaling and unmarshalling into the platform.
Marshaling is present in the @aws-sdk/util-dynamodb module, if that helps.
Yup. The question is do we make that a dependency? We could do that and then require the user to pass in the v3 DynamoDB client like we do currently with the DocumentClient (so they could still instrument it). Of course, as soon as we build that, AWS will release an update with a DocumentClient. 🤦♂️
Given the separate packages, could v2 and v3 coexist and the library auto-detect (or have a new mechanism for passing DDB for v3) which version is being used?
Although this is one of the best libraries out there, the inability to use the latest SDK version that drastically reduces the bundle size for AWS lambda functions block people from installing this package. The upgrade process is not straightforward; however, it is very unpleasant to see such a great product that cannot catch up with the updates AWS even releases NodeJS 14 official support now, so it would be awesome to see this library as a community standard going forward.
The issue with v3 is that there isn’t a DocumentClient, which is what this library uses under the hood. I’ve already built some prototypes that use v3 and apply marshall/unmarshall in the library, but this would create a massive breaking change. I was hoping that v3 would have a DocumentClient before GA, but unfortunately, that didn’t happen.
I can confirm in other projects; we have been using V3 without notable issues. The marshall/unmarshall indeed is the recommended way from AWS ATM. and is the one we have been referring to as well and can say it works without issues.
Would you please let us know what challenges the library is facing so that we as a community might try to help and support you in this migration journey?
I can confirm in other projects; we have been using V3 without notable issues. The
marshall/unmarshallindeed is the recommended way from AWS ATM. and is the one we have been referring to as well and can say it works without issues.Would you please let us know what challenges the library is facing so that we as a community might try to help and support you in this migration journey?
How have you been doing transactions with v3 without the DocumentClient?
V3 has only a single concept and everything comes down to DynamoDBClient and Send Method. for the transaction we used the instance of a TransactWriteItemsCommand which accepts TransactWriteItemsCommandInput
It looks like they (@trivikr) are "prioritising" DocumentClient in v3.
Considering it's popularity in terms of reactions, and the comments on DocumentClient support - we should consider prioritizing it. I'll raise this request in our planning meeting.
Might be worth waiting to see what they do with this before investing any significant effort.
However, I don't know if this defeats the point raised above (taken from the experimental PR for DocumentClient put):
We're not planning to publish modular implementation of Document Client as of now, as that would not be straightforward.
V3 doesn't seem complete yet.
We're not planning to publish modular implementation of Document Client as of now
The modular implementation was added in c29b57d (#2031)
Having said that, this is still an experimental changes and have to be reviewed within the team. Do comment on the PR if you have ideas on anything can be done better or differently.
This PR has been merged today, so v3 got DocumentClient.
https://github.com/aws/aws-sdk-js-v3/pull/2097
There is no update on this topic, even though many other libraries already implemented this. OneTable is already compatible with SDK3, and AWS released an update. Any ETA?
@rati-dzidziguri there's a few issues as mentioned in this thread that definitely increase the complexity on getting this implemented in this library however, I'm sure Jeremy would be happy to review a PR solving this if submitted.
Do we have a plan for this? The implementation in AWS/aws-sdk-js-v3#2097 is not compatible with this library.
I think we should not rely on documentClient and use marshal/unmarshal instead as it is the recommended path forward by AWS.
If we can agree on the path forward for this library, I am happy to implement it and create a PR.
What benefits does v3 bring for us? Bundle size? Is there anything else? Anything we need from v3 that v2 doesn't already do?
Would the effort be better spent getting this library to v1 and then adding support for v3 after?
We haven't got a stable version of this library yet and there is a lot of feature spread which needs fixing prior to v1.
Here is the my example problem statement :
I have a service and I am using eventbridge and dynamodb. I want to use version 3 aws sdk to reduce the bundle size. I used eventbridge module. Now, I want to update to use dynamodb module. But I can't as I am dependent on dynamodb toolbox and it depends on aws sdk.
As we are not yet to version 1 and not to the stable release yet, to me it makes sense to use the optimised version of aws sdk.
Is it just bundle size that is the driver for updating to v3, or is there something else?
Does v2 have bugs that are fixed in v3? Etc.
I agree that it would be nice to use v3 and I agree that using un/marshall is my (totally uneducated and based upon the AWS media releases) preferred choice, however I just want to see this library get to v1 sooner rather than later.
Is there any update? I can see an open PR for a migration to the v3 SDK but it's been open for months now.
@jeremydaly would love to here your thoughts on this. Would be nice to know if this is in roadmap or not.
Yes, this is coming. We've been waiting on our migration to v0.4 that does automatic type inferencing. This should be ready soon.
I like the way that typedorm have implemented this, with an abstraction that allows you to choose v2 or v3. It's quite neat.
@jeremydaly How should I use the v3 client? I am getting type errors
You don't... yet.
How much more time will it take for V4 to be released? And what are the changes we are looking for, I might try to send a PR
How much more time will it take for V4 to be released? And what are the changes we are looking for, I might try to send a PR
It's been in the pipeline for a while, so hopefully we're close! I think the code was done but unmergeable. If you really need to use v3 have a look at TypedORM.
PS, it's v0.4 not v4. We haven't made it to v1 yet. Hopefully we're also close to that!