spark-rapids
spark-rapids copied to clipboard
Fix tests failures in subquery_test.py
FAILED ../../../../integration_tests/src/main/python/subquery_test.py::test_scalar_subquery_struct
FAILED ../../../../integration_tests/src/main/python/subquery_test.py::test_scalar_subquery_struct
FAILED ../../../../integration_tests/src/main/python/subquery_test.py::test_scalar_subquery_struct
FAILED ../../../../integration_tests/src/main/python/subquery_test.py::test_scalar_subquery_array
FAILED ../../../../integration_tests/src/main/python/subquery_test.py::test_scalar_subquery_struct
FAILED ../../../../integration_tests/src/main/python/subquery_test.py::test_scalar_subquery_array
FAILED ../../../../integration_tests/src/main/python/subquery_test.py::test_scalar_subquery_array
FAILED ../../../../integration_tests/src/main/python/subquery_test.py::test_scalar_subquery_struct
FAILED ../../../../integration_tests/src/main/python/subquery_test.py::test_scalar_subquery_struct
FAILED ../../../../integration_tests/src/main/python/subquery_test.py::test_scalar_subquery_struct
FAILED ../../../../integration_tests/src/main/python/subquery_test.py::test_scalar_subquery_struct
FAILED ../../../../integration_tests/src/main/python/subquery_test.py::test_scalar_subquery_array
FAILED ../../../../integration_tests/src/main/python/subquery_test.py::test_scalar_subquery_array
FAILED ../../../../integration_tests/src/main/python/subquery_test.py::test_scalar_subquery_struct
FAILED ../../../../integration_tests/src/main/python/subquery_test.py::test_scalar_subquery_array
FAILED ../../../../integration_tests/src/main/python/subquery_test.py::test_scalar_subquery_array
FAILED ../../../../integration_tests/src/main/python/subquery_test.py::test_scalar_subquery_struct
FAILED ../../../../integration_tests/src/main/python/subquery_test.py::test_scalar_subquery_struct
FAILED ../../../../integration_tests/src/main/python/subquery_test.py::test_scalar_subquery_array
FAILED ../../../../integration_tests/src/main/python/subquery_test.py::test_scalar_subquery_array
FAILED ../../../../integration_tests/src/main/python/subquery_test.py::test_scalar_subquery_map
Some context here: These tests pass when ANSI mode is disabled.
More info: subquery_test.py::test_scalar_subquery_struct fails because of a possible bug in libcudf:
E Suppressed: com.nvidia.spark.rapids.jni.GpuRetryOOM: injected RetryOOM
E at ai.rapids.cudf.ColumnView.hasNonEmptyNulls(Native Method)
E at ai.rapids.cudf.ColumnView.hasNonEmptyNulls(ColumnView.java:5218)
E at ai.rapids.cudf.AssertEmptyNulls.assertNullsAreEmpty(AssertEmptyNulls.java:33)
E at ai.rapids.cudf.ColumnView.<init>(ColumnView.java:58)
...
Somewhere along the way, it appears that the output columns have non-empty null rows. This will need to be fixed in libcudf. I'll raise a separate bug to track this.
Edit: https://github.com/NVIDIA/spark-rapids/issues/11090