dynamodb-toolbox
dynamodb-toolbox copied to clipboard
Custom parse method for attributes
Hi and thanks for this awesome not-ORM 🙂
I was wondering if we could introduce a custom parse
method for Entities' attributes. Basically, I need something that is the opposite of transform
when an item is read from DynamoDB.
It could be useful if you want to store a value differently in DynamoDB than in your application.
A couple of examples:
-
store large texts as gzipped binaries Use
transform
to gzip the value, andparse
to decompress it. -
add/remove padding zeroes Sometimes, you want to store numbers as zero-padded strings. For example, if you want to index them in a string SK. Use
transform
to pad the value andparse
to remove it and transform the value back into a number. -
base64 encode/decode.
I'd be happy to open a PR and I am open to suggestions/ideas.
Thank you!
Hi @bboure. Good thoughts.
The second use case is (kind of) supported using defaults. You need to store the number separately, but default can have a function to pad the string.
Storage is generally cheaper than compute, so this isn't an anti-pattern (data duplication). In fact, it is generally recommended to seperate your data and your keys and avoid parsing.
Your first and last use cases aren't supported out of the box but definitely can be done "manually" in your data access code.
I can see the benefits of a transform
and parse
method for each attribute (though we must remember that storage is cheaper than compute in serverless so you wouldn't want to abuse it).
I do wonder whether this should be a responsibility of this library though. Feature bloat makes it hard to maintain and it would be easy to implement in your data access layer.
Thank you for your feedback @darbio
Storage is generally cheaper than compute, so this isn't an anti-pattern (data duplication). In fact, it is generally recommended to separate your data and your keys and avoid parsing.
You're absolutely right! In fact, this is exactly what I do. I was just trying to find examples to illustrate what I was suggesting :) My real use-case is the first one (gzip/ungzip content). That is because I have potentially very large items that will need to be compressed for cost optimization (less RCU, WCU and storage).
I do wonder whether this should be a responsibility of this library though.
Yes, I was wondering that too, but since there is a transform
method (this is currently where I gzip the content), I was also wondering why there isn't an "untransform" one.
Hi @bboure, I believe this was suggested before, but I like the overall idea.
Hey @bboure, this was released in v0.5 😎
You can now use the format
function to achieve what you wanted.