dask-sql icon indicating copy to clipboard operation
dask-sql copied to clipboard

[DF] Honor column nullability flag from parquet files

Open jdye64 opened this issue 3 years ago • 0 comments

Is your feature request related to a problem? Please describe. When reading tables from Parquet files the "nullability" of the column should be read from the source parquet file. Currently we just hardcode the nullability to false which is inaccurate.

Describe the solution you'd like When registering a table in the DaskSqlContext the nullability of the columns should be passed along in the schema metadata so that the appropriate calls can be made to Field::new(...)

Describe alternatives you've considered None

Additional context None

jdye64 avatar Jun 23 '22 18:06 jdye64