substrait icon indicating copy to clipboard operation
substrait copied to clipboard

ExpandRel output type discrepancy

Open vbarua opened this issue 1 year ago • 1 comments

The protobuf comment

In addition to a field being emitted per input field an extra int64 field is emitted

Disagrees with the written spec:

The expand fields followed by an i32 column describing the index of the duplicate that the row is derived from.

We should reconcile this.

vbarua avatar Sep 26 '24 19:09 vbarua

Should it be "i32 or i64 field", allowing both? At least in Spark the type of the field depends on some config option, defaulting now to i64 but earlier i32: https://github.com/apache/spark/blob/master/docs/sql-migration-guide.md#upgrading-from-spark-sql-30-to-31 (see second item).

Blizzara avatar Nov 08 '24 18:11 Blizzara