spark icon indicating copy to clipboard operation
spark copied to clipboard

[SPARK-49985][SQL] Remove support for interval types in Variant

Open harshmotw-db opened this issue 1 year ago • 1 comments

What changes were proposed in this pull request?

Support for interval types was added to the variant spec. This PR removes this support and removes the ability to cast from interval types to variant and vice versa.

Why are the changes needed?

I implemented interval support for Variant before, but because the Variant spec type is supposed to be open and compatible with other engines which may not support all the ANSI Interval types, more thought needs to be put into the design of these intervals in Variant.

Does this PR introduce any user-facing change?

Yes, after this change, users would no longer be able to cast between variants and intervals.

How was this patch tested?

Unit tests making sure that

  1. It is not possible to construct variants containing intervals.
  2. It is not possible to cast variants to intervals.
  3. Interval IDs in variants are treated just like other unknown type IDs.

Was this patch authored or co-authored using generative AI tooling?

No.

harshmotw-db avatar Sep 23 '24 22:09 harshmotw-db

cc @cashmand For changes in the README file.

harshmotw-db avatar Sep 23 '24 22:09 harshmotw-db

@cloud-fan @HyukjinKwon can you please look at this PR? Thanks!

harshmotw-db avatar Oct 22 '24 18:10 harshmotw-db

Merged to master.

HyukjinKwon avatar Oct 23 '24 00:10 HyukjinKwon