panbingkun

[SPARK-37599][SQL][TESTS] Unify v1 and v2 ALTER TABLE .. SET LOCATION tests

2

### What changes were proposed in this pull request? - Move parser tests from DDLParserSuite to AlterTableSetLocationParserSuite. - Port DS v1 tests from DDLSuite and other test suites to v1.AlterTableSetLocationSuite....

[SPARK-39892][SQL] Use ArrowType.Decimal(precision, scale, bitWidth) instead of ArrowType.Decimal(precision, scale)

1

### What changes were proposed in this pull request? Use ArrowType.Decimal(precision, scale, bitWidth) instead of ArrowType.Decimal(precision, scale) to cleanup following compilation warnings: > [warn] /home/runner/work/spark/spark/sql/catalyst/src/main/scala/org/apache/spark/sql/util/ArrowUtils.scala:48:49: [deprecation @ org.apache.spark.sql.util.ArrowUtils.toArrowType | origin=org.apache.arrow.vector.types.pojo.ArrowType.Decimal....

PARQUET-2156: Column bloom filter: Show bloom filters in tools

2

1.show all column bloom filter, result as follow: > **parquet-tools bloom-filter BloomFilter.snappy.parquet** > row-group 0: bloom filter for column id: NONE > bloom filter for column uuid: Hash strategy: block...

[SPARK-40501][SQL] Enhance 'SpecialLimits' to support project(..., limit(...))

2

### What changes were proposed in this pull request? The pr aim to enhance 'SpecialLimits' to support project(..., limit(...)), for Improve query performance ### Why are the changes needed? When...

[SPARK-39814] Use AmazonKinesisClientBuilder.withCredentials instead of new AmazonKinesisClient(credentials)

5

### What changes were proposed in this pull request? Use AmazonKinesisClientBuilder.withCredentials instead of new AmazonKinesisClient(credentials) to cleanup following compilation warnings: > [warn] /home/runner/work/spark/spark/connector/kinesis-asl/src/main/scala/org/apache/spark/examples/streaming/KinesisWordCountASL.scala:108:25: [deprecation @ org.apache.spark.examples.streaming.KinesisWordCountASL.main.kinesisClient | origin=com.amazonaws.services.kinesis.AmazonKinesisClient. | version=]...

[SPARK-40355][SQL] Improve pushdown for orc & parquet when cast scenario

4

### What changes were proposed in this pull request? #### Table Schema: | a | b | part | |---|---|---| | int | **string** | **string** | #### SQL: select...

[WIP] investigate the root cause for SPARK-40165

1

### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change? ### How was this patch tested?

### What changes were proposed in this pull request? The pr aims to provide better prompts when option's `compression` is null. ### Why are the changes needed? In the original...

[SPARK-46654][SQL] Make to_csv can correctly display complex types data

7

### What changes were proposed in this pull request? The pr aims to fix `to_cvs` displaying complex types (ArrayType, MapType, StructType) incorrectly. ### Why are the changes needed? ``` val...

### What changes were proposed in this pull request? The pr aims to improve the `error prompt` when `SaveMode is null` in API `DataFrameWriter.mode`. ### Why are the changes needed?...