spark-rapids icon indicating copy to clipboard operation
spark-rapids copied to clipboard

Clean the `try ... finally ...` blocks in the `GpuFileFormatDataWriter`

Open HaoYang670 opened this issue 3 years ago • 7 comments

Signed-off-by: remzi [email protected] Close #6615.

Changes in the PR

Replace

var v = null
try {
  v = some_value
  // do something with v
  v.close()
  v = null
} finally {
  if (v != null) {
    v.close()
  }
}

with

withResource(some_value) { v =>
  // do something with v
}
  1. We find that the ParquetWriteSuite doesn't cover all the code branches. So to add more cases to test the impact MaxRecordsPerFile in that suite.

HaoYang670 avatar Oct 08 '22 05:10 HaoYang670

build

HaoYang670 avatar Oct 08 '22 05:10 HaoYang670

build

HaoYang670 avatar Oct 08 '22 07:10 HaoYang670

build

HaoYang670 avatar Oct 08 '22 07:10 HaoYang670

build

HaoYang670 avatar Oct 09 '22 09:10 HaoYang670

build

HaoYang670 avatar Oct 10 '22 09:10 HaoYang670

build

HaoYang670 avatar Oct 10 '22 10:10 HaoYang670

build

HaoYang670 avatar Oct 12 '22 08:10 HaoYang670

build

HaoYang670 avatar Oct 21 '22 01:10 HaoYang670

build

HaoYang670 avatar Oct 28 '22 02:10 HaoYang670

build

HaoYang670 avatar Nov 01 '22 07:11 HaoYang670

Hi, @jlowe, I prefer to drop this PR as it seems like the withReousrce can't make the code cleaner in this context. What's your opinion.

HaoYang670 avatar Nov 14 '22 06:11 HaoYang670

Some of the changes in this PR are fine, it's mostly problematic in the write method (which admittedly is most of the PR). Given the write method needs refactoring to resolve #6980, we can postpone changes to that method for now.

jlowe avatar Nov 14 '22 14:11 jlowe

@HaoYang670 this should be moved to 23.02 as it doesn't make sense to put this into 22.12 at this point in its release cycle.

jlowe avatar Nov 29 '22 21:11 jlowe

this should be moved to 23.02

My apologies, that's already occurred.

jlowe avatar Nov 29 '22 21:11 jlowe