Allison Wang comments

Results 6 comments of


                                            Allison Wang

Randomly getting 'at least one column must be specified for the table'

Hi @benc-db thanks for reporting the issue. Are you using Databricks clusters or SQL endpoints? If you are using clusters, which is the DBR version of your cluster?

[SPARK-48710][PYTHON] Use NumPy 2.0 compatible types

cc @itholic

[SPARK-48493][PYTHON] Enhance Python Datasource Reader with direct Arrow Batch support for improved performance

@LucaCanali please re-trigger the failed tests (they seems unrelated to this change)

[SPARK-48493][PYTHON] Enhance Python Datasource Reader with direct Arrow Batch support for improved performance

Thanks! Merging to master.

[SPARK-48482][PYTHON][FOLLOWUP] dropDuplicates and dropDuplicatesWIthinWatermark should accept named parameter

My general comment here is that we should be opinionated and only have one way to perform certain operations. After this change, users now have two identical ways to drop...

[SPARK-51919][PYTHON] Allow overwriting statically registered Python Data Source

thanks, merging to master