Wenchen Fan comments

Results 245 comments of


                                            Wenchen Fan

[SPARK-48739][SQL] Disable writing collated data to file formats that don't support them in non managed tables

For internal file source API, I think we can simply update `FileFormat#supportDataType` in certain formats such as CSV to return false for string with collation. So no new API is...

[SPARK-48739][SQL] Disable writing collated data to file formats that don't support them in non managed tables

> This would mean that we wouldn't be able to create spark managed tables with collations for those formats. Is that something that we want to do? To confirm the...

[SPARK-48697][SQL] Add collation aware string filters

I think there is some misunderstanding here. Filter pushdown has a few steps: 1. Spark translates catalyst filters to data source filters, which can be a semantically subset as some...

[SPARK-48697][SQL] Add collation aware string filters

thanks, merging to master!

[SPARK-48720][SQL] Fix `IF EXISTS` ignored for `ALTER TABLE ... UNSET … TBLPROPERTIES ...` in v2

Yea UNSET non-existing property should be noop. BTW we should throw table not found error if table does not exist.

[SPARK-48280][SQL] Improve collation testing surface area using expression walking

thanks, merging to master!

[SPARK-48416][SQL] Support related nested WITH expression

Can we solve this issue by recursion? Which is easier to understand.

[SPARK-48416][SQL] Support related nested WITH expression

Sorry for the late review. I think the implementation can be much simpler, please see my attempt at https://github.com/apache/spark/pull/49093

[SPARK-36680][SQL] Supports Dynamic Table Options for Spark SQL

only one comment: https://github.com/apache/spark/pull/46707/files#r1669539861 , I'm also fine to just remove `OPTIONS`.

[SPARK-36680][SQL] Supports Dynamic Table Options for Spark SQL

thanks, merging to master!