Wenchen Fan comments

Results 245 comments of


                                            Wenchen Fan

[SPARK-47009][SQL] Enable create table support for collation

> AFAIK there is no such thing Yes, and we should mention it in the PR description. We still map string with collation to parquet string type. This means we...

[SPARK-47009][SQL] Enable create table support for collation

@stefankandic the new test has failures ``` - add collated column with alter table *** FAILED *** (167 milliseconds) [info] org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in...

[SPARK-47009][SQL] Enable create table support for collation

thanks, merging to master!

[SPARK-47070] Fix invalid aggregation after in-subquery rewrite

> Semantically, we should wrap in any_value(), but any_value() throws RuntimeReplaceableAggregate.aggBufferAttributes should not be called and is not fully supported It's because `any_value` is not executable and need to be...

[SPARK-47070] Fix invalid aggregation after in-subquery rewrite

thanks, merging to master!

[SPARK-46743] Count bug after constant folding

thanks, merging to master!

[SPARK-47015][SQL] Disable partitioning on collated columns

how about bucket columns? We generate the bucket id from the string value and assume all the semantically-same string values should generate the same bucket id, which isn't true for...

[SPARK-45789][SQL] Support DESCRIBE TABLE for clustering columns

thanks, merging to master!

[SPARK-47007][SQL] SortMap function

Sorry I missed this. Why do we add this public function? Do other systems have it? To support GROUP BY map type, an internal `MapSort` expression is sufficient.

[SPARK-47007][SQL] SortMap function

I can't find it in other systems, and it does not make sense as map elements are order-less. I'm reverting it, please re-submit it without exposing the function publicly.