koheesio icon indicating copy to clipboard operation
koheesio copied to clipboard

[FEATURE] Quarantine transformation

Open mikita-sakalouski opened this issue 8 months ago • 0 comments

Is your feature request related to a problem? Please describe.

Quarantine transformation can be used to split your incoming data based on custom logic and write "bad" part to different table, while propagating good part to the next koheesio Step.

Describe the solution you'd like

Should be pretty simple, but customizable pyspark transformation with splitting logic.

Describe alternatives you've considered

We should Delta Live Tables feature for the same issue, which is providing more functionality (metrics, stats and etc.)

mikita-sakalouski avatar May 30 '24 10:05 mikita-sakalouski