spark-rapids
spark-rapids copied to clipboard
Fix AQE test failures on Databricks 14.3
Build the plugin against Databricks 14.3 cluster using #11467. Once built successfully run the aqe tests by TESTS=aqe_test.py jenkins/databricks/test.sh
The following tests fail
[gw0] [ 68%] FAILED ../../src/main/python/aqe_test.py::test_aqe_join_with_dpp
[gw0] [ 68%] FAILED ../../src/main/python/aqe_test.py::test_aqe_join_with_dpp_multi_columns
Taking a look at this one now.
It appears that this is the result of a new, unsupported exec being used to handle BHJs.
It will be beyond the scope of this issue to add support. I'm investigating for details on the exec, before I file the issue and disable these tests.