spark
spark copied to clipboard
apache
[SPARK-48280][SQL] Improve collation testing surface area using expression walking
What changes were proposed in this pull request?
This PR is introducing Expression Walker in different forms in order to improve collation testing surface area. The tests added include:
- Expression Walker for expression evaluation
- Expression Walker for SQL query examples
- Expression Walker for codeGen generation
Why are the changes needed?
Collations introduced a lot of changes to many functions and parts of the code and these tests aim to catch existing errors and prevent addition of new functions without proper implementation of collation support. To emphasise the importance of these tests, some of the relevant tickets that were opened as a byproduct of this testing:
- https://issues.apache.org/jira/browse/SPARK-48472
- https://issues.apache.org/jira/browse/SPARK-48572
- https://issues.apache.org/jira/browse/SPARK-48574
- https://issues.apache.org/jira/browse/SPARK-48600
- https://issues.apache.org/jira/browse/SPARK-48662
Does this PR introduce any user-facing change?
No.
How was this patch tested?
This PR is only related to testing.
Was this patch authored or co-authored using generative AI tooling?
No.
This is now ready for merge, all tests have passed, as find_in_set was fixed this morning.
