datayoga icon indicating copy to clipboard operation
datayoga copied to clipboard

`relational.write` - deduplication mechanism

Open spicy-sauce opened this issue 2 years ago • 0 comments

When we get the same key in the same batch, the whole batch will probably fail. First, add a test that verifies it and fails. Then, add a dedup mechanism that groups the records by the key in the relational.write block and takes only the latest record per key.

spicy-sauce avatar Jan 03 '23 07:01 spicy-sauce