spark-rapids
spark-rapids copied to clipboard
Fix tests failures in join_test.py
FAILED ../../../../integration_tests/src/main/python/join_test.py::test_broadcast_hash_join_fix_fallback_by_inputfile
FAILED ../../../../integration_tests/src/main/python/join_test.py::test_broadcast_nested_join_fix_fallback_by_inputfile
FAILED ../../../../integration_tests/src/main/python/join_test.py::test_broadcast_nested_loop_join_special_case_count
FAILED ../../../../integration_tests/src/main/python/join_test.py::test_broadcast_nested_loop_join_special_case_group_by_count
FAILED ../../../../integration_tests/src/main/python/join_test.py::test_broadcast_nested_loop_join_with_condition
FAILED ../../../../integration_tests/src/main/python/join_test.py::test_cartesian_join_special_case_count
FAILED ../../../../integration_tests/src/main/python/join_test.py::test_cartesian_join_special_case_group_by_count
FAILED ../../../../integration_tests/src/main/python/join_test.py::test_distinct_join
FAILED ../../../../integration_tests/src/main/python/join_test.py::test_half_cache_join
FAILED ../../../../integration_tests/src/main/python/join_test.py::test_left_broadcast_nested_loop_join_condition_missing_count
FAILED ../../../../integration_tests/src/main/python/join_test.py::test_right_broadcast_nested_loop_join_condition_missing_count
Two of these tests (test_broadcast_hash_join_fix_fallback_by_inputfile, and test_broadcast_nested_join_fix_fallback_by_inputfile) seem to be failing with the same missing gpuDecodeTime metric:
E Caused by: java.util.NoSuchElementException: key not found: gpuDecodeTime
E at scala.collection.immutable.Map$EmptyMap$.apply(Map.scala:243)
E at scala.collection.immutable.Map$EmptyMap$.apply(Map.scala:239)
E at com.nvidia.spark.rapids.OrcTableReader.next(GpuOrcScan.scala:2884)
E at com.nvidia.spark.rapids.OrcTableReader.next(GpuOrcScan.scala:2859)
This comes from the ORC file reader.
The rest of the tests seem to run fine, when ANSI mode is disabled.
I've unassigned myself from this bug. This should be resolved after the missing metrics are addressed.