spark icon indicating copy to clipboard operation
spark copied to clipboard

Published 20 hours ago • verified publisher icon apache

[SPARK-40216][SQL] Extract common `ParquetUtils.prepareWrite` method to deduplicate code in `ParquetFileFormat` and `ParquetWrite`

Open LuciferYang opened this issue 2 years ago • 1 comments

What changes were proposed in this pull request?

This pr is a refactor work, the main change is extract ParquetUtils.prepareWrite method to eliminate duplicate code in ParquetFileFormat and ParquetWrite.

Why are the changes needed?

Eliminate duplicate code

Does this PR introduce any user-facing change?

No

How was this patch tested?

Pass GitHub Actions

LuciferYang avatar Aug 25 '22 06:08 LuciferYang

cc @gengliangwang @cloud-fan Do you have time to help review? Thanks

LuciferYang avatar Aug 25 '22 06:08 LuciferYang

@sadikovi can you take a look?

cloud-fan avatar Sep 28 '22 03:09 cloud-fan

Yes, I will take a look shortly, thanks for letting me know.

sadikovi avatar Sep 28 '22 06:09 sadikovi

Thanks @cloud-fan @sadikovi

LuciferYang avatar Sep 28 '22 06:09 LuciferYang

Thanks @sadikovi ~ rebase to keep the code up to date, let's wait for GA

LuciferYang avatar Sep 29 '22 00:09 LuciferYang

friendly ping @cloud-fan , should we merge this one?

LuciferYang avatar Sep 30 '22 08:09 LuciferYang

thanks, merging to master!

cloud-fan avatar Oct 06 '22 07:10 cloud-fan

Thanks @sadikovi @cloud-fan

LuciferYang avatar Oct 06 '22 07:10 LuciferYang