spark-daria icon indicating copy to clipboard operation
spark-daria copied to clipboard

Lenient schema validation

Open skestle opened this issue 3 years ago • 1 comments

(This single commit PR requires the deep DF validation changes) When validating data against a schema, it verifies that all field elements are the same.

However, a data frame column that is specified as non-null should be considered valid against a schema column that allows nulls.

This PR implements that change.

skestle avatar Jun 10 '21 02:06 skestle

This is draft simply because #144 has not been merged.

skestle avatar Jun 10 '21 02:06 skestle