Wenchen Fan comments

Repositories
Issues
Comments

Results 245 comments of


                                            Wenchen Fan

[SPARK-54539][SQL] TaskMetrics add spillTime metrics

do we have similar metrics for SQL plan nodes?

[SPARK-46741][SQL] Cache Table with CTE won't work

thanks, merging to master/4.1!

[SPARK-49162][SQL] Push down date_trunc function

My biggest concern is we translate the Spark `TruncTimestamp` expression to v2 `DATE_TRUNC` function. `DATE_TRUNC` is not in the SQL standard and personally I think it's more reasonable to let...

[SPARK-49893] Respect user schema nullability for file data sources when DSV2 Table is used.

Is this only a problem for file source v2? If yes I think we should just fix file source v2 to respect `LEGACY_RESPECT_NULLABILITY_IN_TEXT_DATASET_CONVERSION`

[SPARK-50033] Add a hint to logical.Aggregate() node

The pyspark failure is unrelated, thanks, merging to master!