Andy Grove comments

Results 657 comments of


                                            Andy Grove

Implement initial version of to_json

I am working on an initial implementation of this and will have a PR up soon. This should make it easy for others to contribute to flesh out the functionality...

Update supported Spark and Java versions in installation guide

The installation guide now lists the specific Java and Scala versions that are supported for each Spark version

abs returns incorrect value in some cases

This is likely a duplicate of https://github.com/apache/datafusion-comet/issues/642

abs returns incorrect value in some cases

Closing this bug because we disabled the feature. Filed https://github.com/apache/datafusion-comet/issues/1890 to re-implement.

Evaluate use of selection vectors in scan-filter-join operations

This paper may have useful information: "Filter Representation in Vectorized Query Execution" https://db.cs.cmu.edu/papers/2021/ngom-damon2021.pdf

signum(0) returns incorrect result

I filed an issue in DataFusion repo: https://github.com/apache/datafusion/issues/11557

signum(0) returns incorrect result

There is now a PR open against DataFusion to add a Postgres-compatible implementation of signum, which is very close to what we need for Spark. https://github.com/apache/datafusion/pull/11580

Comparison between negative zero and false produces incorrect result

We should test with non-negative zero as well

Improve performance of TPC-H q19

Comet is now faster than Spark for this query, and there is no longer a C2R in the Comet plan, so closing this

Improve performance of TPC-H q16

As an experiment, I removed our check where we fall back to Spark for BuildRight with LeftAnti so that the query runs natively (possibly with incorrect results) and performance increased...