finos
Handle filtering null values
Description
If you had a DataTable with a column where value is null, currently cannot specify null values in a filter e.g. where mycolumn = null Null value is a valid usecase so need to review how we handle it.
Need to consider how null values behaves across server and ui for
- Table row - setting and read value
- Type ahead filter
- json parsing
Useful context
RowWithData we don't have a way to tell the difference between column/field not found and column/field having a null value. Both cases would return null with row.get.
Given following cases as data row:
With all three types, the typeahead for filtering on change returns ["null"] With RowWithNull, RowWithEmptyString, the typeahead for filtering on change returns [] With RowWithEmptyString only, the typeahead for filtering on change returns [ "" ]
Some of the findings for further discussion
Problem 1 When accessing the row data (row.get(columnName)), it doesn’t currently always have all the columns that are defined in the table. So no way of telling whether column does not exist or whether that column doesn’t have a value defined. Possible options
- When new row created, always instantiate with all columns even if values are not set and introduce row.getOrThrow(columnName) which throws if requesting for column that does not exist. Need to check how json serialisation behave for rows with column with null data vs column not being there and its impact on the ui
- Introduce a helper function for getting row data that can do the column validation check + cast to correct type for that column (E.g. Table.getValue(row, columnName) ) This will also avoid the caller to cast the row value to correct type + risk of that cast getting out of sync with column type def
Problem 2 Java has primitive types like int, double which cannot be nullable Typescript String, number is not nullable Table Row can be missing data for the column as not defined OR data could be null from the source. Also for string, empty string can mean different things from a null so need to be able to distinguish on display / filter by either value
Do we expect the table rows to always be nullable or should we be enforcing that it matches the column type at point of entry (when data get added/updated to table row) Do we have a usecase where UI need to handle null and if so do we need define a way to distinguish/handle that?
Created this follow-up issue based on our discussion from yesterday: https://github.com/finos/vuu/issues/1344
