Hongbin Ma (Mahone)

# fixed: https://github.com/Kyligence/ClickHouse/issues/161 # can't fix: - [ ] [wishlist] does not support sum distinct

[Gluten-5152][CH]Support Optimize and VACUUM command for clickhouse tables

6

## What changes were proposed in this pull request? (Please fill in changes proposed in this fix) (Fixes: #5152 ) ## How was this patch tested? (Please explain how this...

[CH]Support Optimize and VACUUM command for clickhouse tables

1

### Description Support Optimize and VACUUM command for clickhouse tables. ### Usage the basic usage is: ``` optimize table_name vacuum table_name RETAIN 0 HOURS ``` delta optimize/vacuum parameters, such as...

[WIP] initial PR for the framework resuing Vanila Spark's UTs (borrow from …

5

…Gluten) This PR close https://github.com/NVIDIA/spark-rapids/issues/10745 This PR adds a framework to enabling Vanilla Spark UTs by: 1. Some utility Traits, most importantly RapidsTestsTrait and RapidsSQLTestsTrait for rapids test suite to...

**Describe the bug** Spark UT Framework enabled RapidsJsonSuite (https://github.com/NVIDIA/spark-rapids/pull/10743), with the following test cases explicitly excluded: - [x] Casting long as timestamp: Not a bug after setting correct timezone -...

repartition-based fallback for hash aggregate

18

this PR closes https://github.com/NVIDIA/spark-rapids/issues/8391. this PR add a config called spark.rapids.sql.agg.fallbackAlgorithm to let user decide a sort-based algorithm or repartition-based algorithm to use when agg cannot be done in a...

Optimzing Expand+Aggregate in sqls with many count distinct [WIP]

7

Fixing https://github.com/NVIDIA/spark-rapids/issues/10799. This PR tries to optimize the Expand&Aggregate exec in the first stage of a sql with many count distinct measures. The optimizations in this PR include: 1. Avoid...

parameterize threshold for growth of single pass partial sort agg

1

add a config called spark.rapids.sql.agg.singlePassPartialSort.estimatedGrowthThreshold. (default value 1.1) Even if spark.rapids.sql.agg.singlePassPartialSortEnabled is true, if the estimated growth of size per row is less than this config, single pass partial sort...