datafusion
datafusion copied to clipboard
More properly handle nullability of types/literals in Substrait
Which issue does this PR close?
Extracted from #10531
Builds on top of #10622 so I'll rebase this once it's merged.
Rationale for this change
More closely propagate null-ability through Substrait conversions
What changes are included in this PR?
Handles the nullability of struct fields during Substrait conversions. This isn't perfect; some things are still assumed to just always be nullable (e.g. Literal list elements).
Are these changes tested?
Tested through round-trip unit tests
Are there any user-facing changes?
Nulls are handled more accurately