datafusion-comet icon indicating copy to clipboard operation
datafusion-comet copied to clipboard

Divide by zero produces Infinity instead of NULL

Open andygrove opened this issue 1 year ago • 1 comments

Describe the bug

SQL

SELECT c38, c91, c38 / c91 FROM test0 ORDER BY c38, c91;

Spark Plan

AdaptiveSparkPlan isFinalPlan=true
+- == Final Plan ==
   *(2) Sort [c38#38 ASC NULLS FIRST, c91#91 ASC NULLS FIRST], true, 0
   +- AQEShuffleRead coalesced
      +- ShuffleQueryStage 0
         +- Exchange rangepartitioning(c38#38 ASC NULLS FIRST, c91#91 ASC NULLS FIRST, 200), ENSURE_REQUIREMENTS, [plan_id=42538]
            +- *(1) Project [c38#38, c91#91, (cast(c38#38 as double) / cast(c91#91 as double)) AS (c38 / c91)#45670]
               +- *(1) ColumnarToRow
                  +- FileScan parquet [c38#38,c91#91] Batched: true, DataFilters: [], Format: Parquet, Location: InMemoryFileIndex(1 paths)[file:/home/andy/git/apache/datafusion-comet/fuzz-testing/test0.parquet], PartitionFilters: [], PushedFilters: [], ReadSchema: struct<c38:float,c91:float>
+- == Initial Plan ==
   Sort [c38#38 ASC NULLS FIRST, c91#91 ASC NULLS FIRST], true, 0
   +- Exchange rangepartitioning(c38#38 ASC NULLS FIRST, c91#91 ASC NULLS FIRST, 200), ENSURE_REQUIREMENTS, [plan_id=42524]
      +- Project [c38#38, c91#91, (cast(c38#38 as double) / cast(c91#91 as double)) AS (c38 / c91)#45670]
         +- FileScan parquet [c38#38,c91#91] Batched: true, DataFilters: [], Format: Parquet, Location: InMemoryFileIndex(1 paths)[file:/home/andy/git/apache/datafusion-comet/fuzz-testing/test0.parquet], PartitionFilters: [], PushedFilters: [], ReadSchema: struct<c38:float,c91:float>

Comet Plan

AdaptiveSparkPlan isFinalPlan=true
+- == Final Plan ==
   *(2) Sort [c38#38 ASC NULLS FIRST, c91#91 ASC NULLS FIRST], true, 0
   +- AQEShuffleRead coalesced
      +- ShuffleQueryStage 0
         +- Exchange rangepartitioning(c38#38 ASC NULLS FIRST, c91#91 ASC NULLS FIRST, 200), ENSURE_REQUIREMENTS, [plan_id=42603]
            +- *(1) ColumnarToRow
               +- CometProject [c38#38, c91#91, (c38 / c91)#45678], [c38#38, c91#91, (cast(c38#38 as double) / cast(c91#91 as double)) AS (c38 / c91)#45678]
                  +- CometScan parquet [c38#38,c91#91] Batched: true, DataFilters: [], Format: CometParquet, Location: InMemoryFileIndex(1 paths)[file:/home/andy/git/apache/datafusion-comet/fuzz-testing/test0.parquet], PartitionFilters: [], PushedFilters: [], ReadSchema: struct<c38:float,c91:float>
+- == Initial Plan ==
   Sort [c38#38 ASC NULLS FIRST, c91#91 ASC NULLS FIRST], true, 0
   +- Exchange rangepartitioning(c38#38 ASC NULLS FIRST, c91#91 ASC NULLS FIRST, 200), ENSURE_REQUIREMENTS, [plan_id=42583]
      +- CometProject [c38#38, c91#91, (c38 / c91)#45678], [c38#38, c91#91, (cast(c38#38 as double) / cast(c91#91 as double)) AS (c38 / c91)#45678]
         +- CometScan parquet [c38#38,c91#91] Batched: true, DataFilters: [], Format: CometParquet, Location: InMemoryFileIndex(1 paths)[file:/home/andy/git/apache/datafusion-comet/fuzz-testing/test0.parquet], PartitionFilters: [], PushedFilters: [], ReadSchema: struct<c38:float,c91:float>

First difference at row 15: Spark: -3.4028235E38,-0.0,NULL Comet: -3.4028235E38,-0.0,Infinity

Steps to reproduce

No response

Expected behavior

No response

Additional context

No response

andygrove avatar Jul 15 '24 15:07 andygrove

Will be addressed with https://github.com/apache/datafusion-comet/pull/585

kazuyukitanimura avatar Jul 17 '24 22:07 kazuyukitanimura