datafusion icon indicating copy to clipboard operation
datafusion copied to clipboard

More properly handle nullability of types/literals in Substrait

Open Blizzara opened this issue 9 months ago • 1 comments

Which issue does this PR close?

Extracted from #10531

Builds on top of #10622 so I'll rebase this once it's merged.

Rationale for this change

More closely propagate null-ability through Substrait conversions

What changes are included in this PR?

Handles the nullability of struct fields during Substrait conversions. This isn't perfect; some things are still assumed to just always be nullable (e.g. Literal list elements).

Are these changes tested?

Tested through round-trip unit tests

Are there any user-facing changes?

Nulls are handled more accurately

Blizzara avatar May 23 '24 16:05 Blizzara