spark icon indicating copy to clipboard operation
spark copied to clipboard

Published 20 hours ago • verified publisher icon apache

[SPARK-40314][SQL][PYTHON] Add scala and python bindings for inline and inline_outer

Open Kimahriman opened this issue 2 years ago • 1 comments

What changes were proposed in this pull request?

Adds Scala and Python bindings for SQL functions inline and inline_outer

Why are the changes needed?

Currently these functions can only be used via SQL or through expr functions. This makes it a little easier to use them with the DataFrame APIs.

Does this PR introduce any user-facing change?

Exposes new functions directly instead of only through SQL.

How was this patch tested?

Updated existing inline tests to use the new Scala binding instead of being called through SQL expressions

Kimahriman avatar Sep 02 '22 11:09 Kimahriman

Can one of the admins verify this patch?

AmplabJenkins avatar Sep 04 '22 03:09 AmplabJenkins

also, what about adding some tests in python/pyspark/sql/tests/test_functions.py?

zhengruifeng avatar Sep 27 '22 08:09 zhengruifeng

also, what about adding some tests in python/pyspark/sql/tests/test_functions.py?

Thought I found all the places there were explode tests to add inline as well but missed these somehow, I'll add one

Kimahriman avatar Sep 27 '22 20:09 Kimahriman

Merged into master, thanks all!

zhengruifeng avatar Sep 30 '22 01:09 zhengruifeng