data-curator
data-curator copied to clipboard
qcif
Support missing values at field level
Desired Behaviour
Support missingValues at field level - possible changes in specification. See https://github.com/frictionlessdata/specs/issues/551
- Add property to column
- Use property in validation
Note this PR https://github.com/frictionlessdata/specs/pull/588 for "no missing values".
Need to think about the UI for this
- table vs column level
- explaining the difference between the
requiredconstraint and nomissingValues
/remind me on april 24 to write the acceptance test for:
- Set Column and Table properties
- Validate Table
![reminders[bot] avatar](https://avatars.githubusercontent.com/u/26350515?v=4 "Published by a pub.dev verified publisher"){kind=small}
User interface thoughts:
Use missing values ui from Table Properties on Column Properties
Update tooltips to reflect if values apply to Table or Column.
Explain the Column missing values inherit from the Table values
Explain that missing value characters will be interpreted as null
Validate different missing values by column acceptance test done
https://app.cucumber.pro/projects/data-curator/documents/branch/develop/test/features/tools/validate-table.feature#scenario-set-different-missing-values-across-columns
@mattRedBox just realised that this isn't part of the spec yet or even a pattern. I can submit a PR for a pattern. I think you said tableschema.js already supports this - correct?
Proposal https://discuss.okfn.org/t/missing-values-per-field-pattern/6571
PR https://github.com/frictionlessdata/specs/pull/608
Hi @Stephen-Gates They have the variable in place. At the moment, each field/column has a reference to the table level missingValues. We don't copy through this reference yet, but I guess once frictionless head in that direction, it will make it simpler for frictionless to refactor so that the missingValue is specific to each field. We will then need to update what we do.
Hi @Stephen-Gates So reading https://discuss.okfn.org/t/missing-values-per-field-pattern/6571, it reads correctly to me.
Open Knowledge Forums
Following on from Issues #551 in the specs, I’ve decided to propose a new pattern to add to the Frictionless Data Patterns. Looking forward to your feedback 😄 Missing values per field Overview Characters representing missing values in a table can be defined for all fields in a Tabular Data Resource using the missingValues property in a Table Schema. Values that match the missingValues are treated as null. The Missing values per field pattern allows different missing values to be specifie...