datafusion-comet
datafusion-comet copied to clipboard
Fallback reason missing for incompatible casts
Describe the bug
I see a HashAggregate falling back to Spark but the root cause is hidden:
HashAggregate [COMET: Unsupported result expressions found in: List((0.2 * cast((avg(UnscaledValue(l_quantity#130))#124 / 100.0) as decimal(15,6))) AS (0.2 * avg(l_quantity))#125, l_partkey#127L)]
After adding some debug logging I discover the root cause:
Comet does not guarantee correct results for cast from DoubleType to DecimalType(15,6) with timezone Some(America/Denver) and evalMode LEGACY (There can be rounding differences). To enable all incompatible casts, set spark.comet.cast.allowIncompatible=true
I should not need to add debug logging to discover this.
Steps to reproduce
No response
Expected behavior
No response
Additional context
No response