cobrix
cobrix copied to clipboard
Add an error column containing decoding errors of each record
Background
Spark data sources, like csv, create a column containing bogus records it was unable to parse.
Currently, spark-cobol just ignores all decoding errors and writes null to the target fields.
Feature
It would be great if spark-cobol generated a column decoring_errors, containing all fields and values it failed to decode.
This should be optional (with the default being false) since generating this column can have an impact on performance and the output size.