Wenchen Fan comments

Results 245 comments of


                                            Wenchen Fan

[SPARK-39217][SQL] Makes DPP support the pruning side has Union

sorry for the late review, the change looks reasonable to me.

[SPARK-40488] Do not wrap exceptions thrown when datasource write fails

thanks, merging to master!

[SPARK-40473][SQL] Migrate parsing errors onto error classes

Can we update `core/src/main/resources/error/README.md` to mention this special error class naming prefix?

[SPARK-40473][SQL] Migrate parsing errors onto error classes

@dongjoon-hyun @viirya @HyukjinKwon The new error class name prefix `_LEGACY_ERROR_TEMP_` proposed here kind of marks the error as developer-facing, not user-facing. Developers can still get the error class programmatically via...

[SPARK-40045][SQL]Optimize the order of filtering predicates

cc @sigmod

[SPARK-40501][SQL] Enhance 'SpecialLimits' to support project(..., limit(...))

I'm wondering why `PushProjectThroughLimit` does not optimize your query. It should push project through limit.

[SPARK-40501][SQL] Enhance 'SpecialLimits' to support project(..., limit(...))

`PushProjectThroughLimit` is already in the optimizer, or did I miss something?

[SPARK-40501][SQL] Enhance 'SpecialLimits' to support project(..., limit(...))

Ah sorry I misread the code. Let's add this rule then. I think it's beneficial, as it kinds of "normalize" the order of project and limit operator, so that we...

[SPARK-40501][SQL] Enhance 'SpecialLimits' to support project(..., limit(...))

thanks, merging to master!

[SPARK-40501][SQL] Enhance 'SpecialLimits' to support project(..., limit(...))

This is a perf improvement, usually we don't backport.