substrait
substrait copied to clipboard
ExpandRel output type discrepancy
The protobuf comment
In addition to a field being emitted per input field an extra int64 field is emitted
Disagrees with the written spec:
The expand fields followed by an i32 column describing the index of the duplicate that the row is derived from.
We should reconcile this.
Should it be "i32 or i64 field", allowing both? At least in Spark the type of the field depends on some config option, defaulting now to i64 but earlier i32: https://github.com/apache/spark/blob/master/docs/sql-migration-guide.md#upgrading-from-spark-sql-30-to-31 (see second item).