iceberg
iceberg copied to clipboard
ORC: validate values are not null for required columns
The SparkOrcWriter defines converters that could easily throw exceptions when a required column has a null value, like this:
static class RequiredIntConverter implements Converter {
public void addValue(int rowId, int column, SpecializedGetters data,
ColumnVector output) {
if (data.isNullAt(column)) {
throw new NullPointerException("Column " + column + " is required, but null in row: " + rowId);
} else {
output.isNull[rowId] = false;
((LongColumnVector) output).vector[rowId] = data.getInt(column);
}
}
}