spark icon indicating copy to clipboard operation
spark copied to clipboard

[SPARK-49995][SQL] Add named argument support to more TVFs

Open ueshin opened this issue 1 year ago • 1 comments

What changes were proposed in this pull request?

Add named argument support to more TVFs.

SELECT * FROM inline(input => array(struct(1, 'a'), struct(2, 'b')));
SELECT * FROM inline_outer(input => array(struct(1, 'a'), struct(2, 'b')));

SELECT * FROM posexplode(collection => array(1, 2));
SELECT * FROM posexplode_outer(collection => map('a', 1, 'b', 2));

SELECT * FROM variant_explode(input => parse_json('["hello", "world"]'));
SELECT * FROM variant_explode_outer(input => parse_json('{"a": true, "b": 3.14}'));

Why are the changes needed?

The following TVFs should support named argument as same as explode:

  • inline
  • posexplode
  • variant_explode

and their _outer variations.

Does this PR introduce any user-facing change?

Yes, the above functions support named arguments.

How was this patch tested?

Added the related tests.

Was this patch authored or co-authored using generative AI tooling?

No.

ueshin avatar Oct 16 '24 21:10 ueshin

cc @dtenedor @chenhao-db @srielau

ueshin avatar Oct 16 '24 21:10 ueshin

I'dl merge this as-is to be consistent with #48603. @dtenedor Please feel free to leave a comment on the above. Thanks.

ueshin avatar Nov 05 '24 20:11 ueshin

Thanks. merging to master.

ueshin avatar Nov 05 '24 20:11 ueshin