sdk
sdk copied to clipboard
Feat: add the `json` module to stream maps context
Per
- #1500
Specifically, json.loads() and json.dumps() would be helpful.
@aaronsteers Do we need all members of json or can we do with just json.loads and json.dumps?
@aaronsteers Do we need all members of
jsonor can we do with justjson.loadsandjson.dumps?
Good question. Want to add thoughts in the PR?
I like adding json because it's simplest and doesn't require retraining from what users would expect. But I don't mind if we want to implement in another way - especially if for security reasons we want to keep to "safe" implementations only.
Note:
For md5, there just was no good inline way to calculate hashes, so we had to create our own custom function. With json library, the inline method works, so we don't strictly need to do as much in this case.
This has been marked as stale because it is unassigned, and has not had recent activity. It will be closed after 21 days if no further activity occurs. If this should never go stale, please add the evergreen label, or request that it be added.
Still relevant
Things to keep in mind for anyone that's interested in contributing this:
- How to communicate schema types? Perhaps aliases of
json.loadsshould be used, which the SDK can identify as specific types, e.g.parse_json_object,parse_json_array, etc. - See https://github.com/meltano/sdk/blob/97d97de313511be4c21b842bf2ba0283d4bf5c02/singer_sdk/mapper.py#L297-L307